The Library

Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines

Tools

Marshall, A. (Andrea), Altman, Douglas G., Holder, Roger L. and Royston, Patrick. (2009) Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Medical Research Methodology, Vol.9 (Articl). ISSN 1471-2288

Preview

PDF
WRAP_Marshall_Combining.pdf - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (267Kb)

Official URL: http://dx.doi.org/10.1186/1471-2288-9-57

Abstract

Background: Multiple imputation (MI) provides an effective approach to handle missing covariate data within prognostic modelling studies, as it can properly account for the missing data uncertainty. The multiply imputed datasets are each analysed using standard prognostic modelling techniques to obtain the estimates of interest. The estimates from each imputed dataset are then combined into one overall estimate and variance, incorporating both the within and between imputation variability. Rubin's rules for combining these multiply imputed estimates are based on asymptotic theory. The resulting combined estimates may be more accurate if the posterior distribution of the population parameter of interest is better approximated by the normal distribution. However, the normality assumption may not be appropriate for all the parameters of interest when analysing prognostic modelling studies, such as predicted survival probabilities and model performance measures. Methods: Guidelines for combining the estimates of interest when analysing prognostic modelling studies are provided. A literature review is performed to identify current practice for combining such estimates in prognostic modelling studies. Results: Methods for combining all reported estimates after MI were not well reported in the current literature. Rubin's rules without applying any transformations were the standard approach used, when any method was stated. Conclusion: The proposed simple guidelines for combining estimates after MI may lead to a wider and more appropriate use of MI in future prognostic modelling studies.

Item Type:	Journal Article
Subjects:	R Medicine > R Medicine (General) Q Science > QA Mathematics
Divisions:	Faculty of Medicine > Warwick Medical School
Library of Congress Subject Headings (LCSH):	Multiple imputation (Statistics), Missing observations (Statistics), Statistical hypothesis testing -- Asymptotic theory, Mathematical models -- Evaluation, Medical statistics
Journal or Publication Title:	BMC Medical Research Methodology
Publisher:	BioMed Central Ltd.
ISSN:	1471-2288
Date:	28 July 2009
Volume:	Vol.9
Number:	Articl
Identification Number:	10.1186/1471-2288-9-57
Status:	Peer Reviewed
Access rights to Published version:	Open Access
Funder:	Cancer Research UK (CRUK)
References:	1. Altman DG, Royston P: What do we mean by validating a prognostic model? Statistics in Medicine 2000, 19(4):453-473. 2. Wyatt JC, Altman DG: Commentary: Prognostic models: clinically useful or quickly forgotten? British Medical Journal 1995, 311(7019):1539-1541. 3. Burton A, Altman DG: Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines. British Journal of Cancer 2004, 91(1):4-8. 4. Rubin DB: Multiple Imputation for Nonresponse in Surveys New York: John Wiley and Sons; 2004. 5. Graham JW, Olchowski AE, Gilreath TD: How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science 2007, 8(3):206-213. 6. Kenward MG, Carpenter J: Multiple imputation: current perspectives. Statistical Methods in Medical Research 2007, 16(3):199-218. 7. van Buuren S, Boshuizen HC, Knook DL: Multiple imputation of missing blood pressure covariates in survival analysis. Statistics in Medicine 1999, 18(6):681-694. 8. Li KH, Meng XL, Raghunathan TE, Rubin DB: Significance levels from repeated p-values with multiply-imputed data. Statistica Sinica 1991, 1(1):65-92. 9. Schafer JL: Analysis of Incomplete Multivariate Data New York: Chapman and Hall; 1997. 10. Rubin DB, Schenker N: Multiple imputation in health-care databases: an overview and some applications. Statistics in Medicine 1991, 10(4):585-598. 11. Rubin DB, Schenker N: Multiple imputation for interval estimation from simple random samples with ignorable nonresponse. Journal of the American Statistical Association 1986, 81(394):366-374. 12. Hampel FR, Ronchetti EM, Rousseeuw PJ, Stahel WA: Robust statistics. The approach based on influence functions New York: John Wiley & Sons; 1986. 13. Ambler G, Brady AR, Royston P: Simplifying a prognostic model: a simulation study based on clinical data. Statistics in Medicine 2002, 21(24):3803-3822. 14. Peduzzi P, Concato J, Feinstein AR, Holford TR: Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. Journal of Clinical Epidemiology 1995, 48(12):1503-1510. 15. Harrell FE: Regression Modeling Strategies with Applications to Linear Models, Logistic Regression, and Survival Analysis New York: Springer- Verlag; 2001. 16. Schemper M, Stare J: Explained variation in survival analysis. Statistics in Medicine 1996, 15(19):1999-2012. 17. Schemper M, Henderson R: Predictive accuracy and explained variation in Cox regression. Biometrics 2000, 56(1):249-255. 18. O'Quigley J, Xu RH, Stare J: Explained randomness in proportional hazards models. Statistics in Medicine 2005, 24(3):479-489. 19. Harrell FE, Lee KL, Mark DB: Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine 1996, 15(4):361-387. 20. Royston P, Sauerbrei W: A new measure of prognostic separation in survival data. Statistics in Medicine 2004, 23(5):723-748. 21. van Houwelingen HC, Le Cessie S: Predictive value of statistical models. Statistics in Medicine 1990, 9(1):1303-1325. 22. Meng XL, Rubin DB: Performing likelihood ratio tests with multiply-imputed data sets. Biometrika 1992, 79(1):103-111. 23. Fisher RA: Statistical Methods for Research Workers Edinburgh: Oliver and Boyd Ltd; 1941. 24. Hosmer DW, Lemeshow S: Applied survival analysis – Regression modeling of time to event data New York: John Wiley & Sons; 1999. 25. Collett D: Modelling survival data in medical research Second edition. London: Chapman & Hall/CRC; 2003. 26. Thomsen BL, Keiding N, Altman DG: A note on the calculation of expected survival, illustrated by the survival of liver transplant patients. Statistics in Medicine 1991, 10(5):733-738. 27. Clark TG, Altman DG: Developing a prognostic model in the presence of missing data. an ovarian cancer case study. Journal of Clinical Epidemiology 2003, 56(1):28-37. 28. Sinharay S, Stern HS, Russell D: The use of multiple imputation for the analysis of missing data. Psychological Methods 2001, 6(4):317-329. 29. Gill S, Loprinzi CL, Sargent DJ, Thome SD, Alberts SR, Haller DG, Benedetti J, Francini G, Shepherd LE, Seitz JF, et al.: Pooled analysis of fluorouracil-based adjuvant therapy for stage II and III colon cancer: Who benefits and by how much? Journal of Clinical Oncology 2004, 22(10):1797-1806. 30. Clark TG, Stewart ME, Altman DG, Gabra H, Smyth JF: A prognostic model for ovarian cancer. British Journal of Cancer 2001, 85(7):944-952. 31. Rouxel A, Hejblum G, Bernier MO, Boelle PY, Menegaux F, Mansour G, Hoang C, Aurengo A, Leenhardt L: Prognostic factors associated with the survival of patients developing loco-regional recurrences of differentiated thyroid carcinomas. J Clin Endocrinol Metab 2004, 89(11):5362-5368. 32. Stadler WM, Huo DZ, George C, Yang XM, Ryan CW, Karrison T, Zimmerman TM, Vogelzang NJ: Prognostic factors for survival with gemcitabine plus 5-fluorouracil based regimens for metastatic renal cancer. Journal of Urology 2003, 170(4):1141-1145. 33. Vaughn G, Detels R: Protease inhibitors and cardiovascular disease: analysis of the Los Angeles County adult spectrum of disease cohort. AIDS Care 2007, 19(4):492-499. 34. Orsini N, Mantzoros CS, Wolk A: Association of physical activity with cancer incidence, mortality, and survival: a populationbased study of men. British Journal of Cancer 2008, 98(11):1864-1869. 35. Mertens AC, Yasui Y, Neglia JP, Potter JD, Nesbit ME, Ruccione K, Smithson WA, Robison LL: Late mortality experience in fiveyear survivors of childhood and adolescent cancer: The childhood cancer survivor study. Journal of Clinical Oncology 2001, 19(13):3163-3172. 36. Serrat C, Gomez G, de Olalla PG, Cayla JA: CD4+ lymphocytes and tuberculin skin test as survival predictors in pulmonary tuberculosis HIV-infected patients. International Journal of Epidemiology 1998, 27(4):703-712. 37. Bärnighausen T, Tanser F, Gqwede Z, Mbizana C, Herbst K, Newell M-L: High HIV incidence in a community with high HIV prevalence in rural South Africa: findings from a prospective population- based study. AIDS 2008, 22(1):139-144. 38. Harel O: The estimation of R^2 and adjusted R^2 in incomplete data sets using multiple imputation. Journal of Applied Statistics 2009 in press. http://www.informaworld.com/10.1080/ 02664760802553000 39. Heymans MW, van Buuren S, Knol DL, van Mechelen W, de Vet HCW: Variable selection under multiple imputation using the bootstrap in a prognostic study. BMC Medical Research Methodology 2007, 7:33. 40. Hoeting JA, Madigan D, Raftery AE, Volinsky CT: Bayesian model averaging: A tutorial. Statistical Science 1999, 14(4):382-401.
URI:	http://wrap.warwick.ac.uk/id/eprint/2152