Objective: The role of classical and Bayesian statistical approaches remains in dispute in health services research and policy. The goal of this study was to determine if results differ when both analytic techniques are used with the same data set.

Design: We searched MEDLINE and related databases for English-language articles published January 1, 1978 through August 31, 1999. We combined Bayesian and classical statistics search terms and their variants with randomized control trials (RCTs) and meta-analyses.

Results: Searches found 18 studies in 14 publications that met all review criteria—nine RCTs, eight meta-analyses, and one epidemiologic estimate. Statistical analyses using both methods agreed in five RCTs , four meta-analyses, and for the epidemiologic estimates. For four RCTs where results disagreed, classical analysis found the experimental intervention was efficacious compared with the control, and Bayesian reanalysis concluded the intervention was not proven efficacious. Classical meta-analyses of the four studies where results disagreed concluded the experimental intervention was not better than the control; Bayesian reanalysis concluded it was efficacious.

Conclusion: Classical and Bayesian methods in this review exhibited important divergence of results. Disagreement on many fundamental beliefs between classical and Bayesian statistics means continuing debate. One way to resolve this debate is for proponents of each technique to decide together the circumstances for use of each method and analytic framework. If the experts do not agree on the methodologic requirements, other decision makers likely will force their own views.