Analytic perspective

Assessing observational studies of medical treatments

Arthur Hartz¹ , Suzanne Bentler² , Mary Charlton³ , Douglas Lanska⁴ , Yogita Butani⁵ , G Mustafa Soomro⁶ and Kjell Benson⁷

¹  University of Iowa, College of Medicine, Department of Family Medicine, Iowa City, IA 52242 USA

²  University of Iowa, College of Medicine, Department of Family Medicine, Iowa City, IA 52242 USA

³  University of Iowa, College of Medicine, Department of Family Medicine, Iowa City, IA 52242 USA

⁴  VA Medical Center, 500 East Veterans Street, Tomah, WI 54660 USA

⁵  University of Iowa, College of Medicine, Department of Family Medicine, Iowa City, IA 52242 USA

⁶  Section of Community Psychiatry, St. George's Hospital Medical School, London, UK

⁷  Family Practice Clinic, North Colorado Medical Center, Greeley, Colorado USA

author email corresponding author email

Emerging Themes in Epidemiology 2005, 2:8doi:10.1186/1742-7622-2-8

Published:	1 September 2005

Abstract

Background

Previous studies have assessed the validity of the observational study design by comparing results of studies using this design to results from randomized controlled trials. The present study examined design features of observational studies that could have influenced these comparisons.

Methods

To find at least 4 observational studies that evaluated the same treatment, we reviewed meta-analyses comparing observational studies and randomized controlled trials for the assessment of medical treatments. Details critical for interpretation of these studies were abstracted and analyzed qualitatively.

Results

Individual articles reviewed included 61 observational studies that assessed 10 treatment comparisons evaluated in two studies comparing randomized controlled trials and observational studies. The majority of studies did not report the following information: details of primary and ancillary treatments, outcome definitions, length of follow-up, inclusion/exclusion criteria, patient characteristics relevant to prognosis or treatment response, or assessment of possible confounding. When information was reported, variations in treatment specifics, outcome definition or confounding were identified as possible causes of differences between observational studies and randomized controlled trials, and of heterogeneity in observational studies.

Conclusion

Reporting of observational studies of medical treatments was often inadequate to compare study designs or allow other meaningful interpretation of results. All observational studies should report details of treatment, outcome assessment, patient characteristics, and confounding assessment.