Hostname: page-component-8448b6f56d-m8qmq Total loading time: 0 Render date: 2024-04-24T17:15:17.923Z Has data issue: false hasContentIssue false
  • English
  • Français

The importance of accounting for the uncertainty of published prognostic model estimates

Published online by Cambridge University Press:  01 November 2004

Tracey A. Young  and
Simon Thompson
Tracey A. Young
Affiliation:
Brunel University
Simon Thompson
Affiliation:
Institute of Public Health, Cambridge
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives: Reported is the importance of properly reflecting uncertainty associated with prognostic model estimates when calculating the survival benefit of a treatment or technology, using liver transplantation as an example.

Methods: Monte Carlo simulation techniques were used to account for the uncertainty of prognostic model estimates using the standard errors of the regression coefficients and their correlations. These methods were applied to patients with primary biliary cirrhosis undergoing liver transplantation using a prognostic model from a historic cohort who did not undergo transplantation. The survival gain over 4 years from transplantation was estimated.

Results: Ignoring the uncertainty in the prognostic model, the estimated survival benefit of liver transplantation was 16.7 months (95 percent confidence interval [CI], 13.5 to 20.1), and was statistically significant (p<.001). After adjusting for model uncertainty using the standard errors of the regression coefficients, the estimated survival benefit was 17.5 months (95 percent CI, −3.9 to 38.5) and was no longer statistically significant. An additional adjustment for the correlation between regression coefficients widened the 95 percent confidence interval slightly: the estimated survival benefit was 17.0 months (95 percent CI: −4.6 to 38.6).

Conclusions: It is important that the precision of regression coefficients is available for users of published prognostic models. Ignoring this additional information substantially underestimates uncertainty, which can then impact misleadingly on policy decisions.

Keywords

UncertaintySurvival analysisProportional hazards modelsLiver transplantation
Type
GENERAL ESSAYS
Information
International Journal of Technology Assessment in Health Care , Volume 20 , Issue 4 , November 2004 , pp. 481 - 487
Copyright
© 2004 Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Altman DG, Royston P. 2000 What do we mean by validating a prognostic model? Stat Med. 19: 453– 473.Google Scholar
Black N. 1996 Why we need observational studies to evaluate the effectiveness of health care. BMJ. 312: 1215– 1218.Google Scholar
Bonsel GJ, Klompmaker IJ, van 'T Veer F, et al. 1990 Use of prognostic models for assessment of value of liver transplantation in primary biliary cirrhosis. Lancet. 335: 493– 497.Google Scholar
Christensen E, Altman DG, Neuberger J, et al. 1993 Updating prognosis in primary biliary cirrhosis using a time dependent Cox regression model. Gastroenterology. 105: 865– 1876.Google Scholar
Collett D. 1994 Modelling survival data in medical research. Texts in Statistical Science. London: Chapman & Hall;
Decisioneering, Inc. 1994 Crystal ball: User's guide. Denver, CO; Decisioneering, Inc;
Dickson ER, Grambsch PM, Fleming TR, et al. 1989 Prognosis of primary biliary cirrhosis: Model for decision making. Hepatology. 10: 1– 7.Google Scholar
Draper D. 1995 Assessment and propagation of model uncertainty (with discussion) J R Stat Soc B. 57: 45– 97.Google Scholar
Longworth L, Young T, Buxton MJ, et al. 2003 Mid-term cost-effectiveness of the liver transplantation programme of England and Wales for three disease groups. Liver Transpl. 9: 1295– 1307.Google Scholar
Murtaugh PA, Dickson ER, Van Dam GM, et al. 1994 Primary biliary cirrhosis: Prediction of short-term survival based on repeated patient visits. Hepatology. 20: 126– 134.Google Scholar
Neuberger J, Altman DG, Christensen E, Tygstrup N, Williams R. 1986 Use of a prognostic index in evaluation of liver transplantation for primary biliary cirrhosis. Transplantation. 4: 713– 716.Google Scholar
Neuberger J, Lucey MR. 1994 Liver transplantation: Practice and management. Annapolis Junction, MD: BMJ Publishing Group;
Pasha TM, Dickson ER. 1994 Survival algorithms and outcome analysis in primary biliary cirrhosis: Prediction of short-term survival based on repeated patient visits. Hepatology. 20: 126– 134.Google Scholar
Powe NR, Griffiths RI. 1995 The clinical-economic trial: Promise, problems, and challenges. Control Clin Trials. 16: 377– 394.Google Scholar
Spiegelhalter DJ, Thomas A, Best NG. 1999 WinBUGS Version 1.2 User Manual;
Mathsoft Data Analysis Products Division. 2000 S-PLUS 2000 Guide to statistics. vol. 1 and 2. Seattle, WA: Mathsoft Data Analysis Products Division;