The Library

Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines

Tools

Marshall, A. (Andrea), Altman, Douglas G., Holder, Roger L. and Royston, Patrick. (2009) Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Medical Research Methodology, Vol.9 (Articl). ISSN 1471-2288

Preview

PDF
WRAP_Marshall_Combining.pdf - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (267Kb)

Official URL: http://dx.doi.org/10.1186/1471-2288-9-57

Abstract

Background: Multiple imputation (MI) provides an effective approach to handle missing covariate
data within prognostic modelling studies, as it can properly account for the missing data
uncertainty. The multiply imputed datasets are each analysed using standard prognostic modelling
techniques to obtain the estimates of interest. The estimates from each imputed dataset are then
combined into one overall estimate and variance, incorporating both the within and between
imputation variability. Rubin's rules for combining these multiply imputed estimates are based on
asymptotic theory. The resulting combined estimates may be more accurate if the posterior
distribution of the population parameter of interest is better approximated by the normal
distribution. However, the normality assumption may not be appropriate for all the parameters of
interest when analysing prognostic modelling studies, such as predicted survival probabilities and
model performance measures.
Methods: Guidelines for combining the estimates of interest when analysing prognostic modelling
studies are provided. A literature review is performed to identify current practice for combining
such estimates in prognostic modelling studies.
Results: Methods for combining all reported estimates after MI were not well reported in the
current literature. Rubin's rules without applying any transformations were the standard approach
used, when any method was stated.
Conclusion: The proposed simple guidelines for combining estimates after MI may lead to a wider
and more appropriate use of MI in future prognostic modelling studies.

Item Type:

Journal Article

Subjects:

R Medicine > R Medicine (General)
Q Science > QA Mathematics

Divisions:

Faculty of Medicine > Warwick Medical School

Library of Congress Subject Headings (LCSH):

Multiple imputation (Statistics), Missing observations (Statistics), Statistical hypothesis testing -- Asymptotic theory, Mathematical models -- Evaluation, Medical statistics

Journal or Publication Title:

BMC Medical Research Methodology

Publisher:

BioMed Central Ltd.

ISSN:

1471-2288

Official Date:

28 July 2009

Dates:

Date	Event
28 July 2009	Published

Volume:

Vol.9

Number:

Articl

Identification Number:

10.1186/1471-2288-9-57

Status:

Peer Reviewed

Access rights to Published version:

Open Access

Funder:

Cancer Research UK (CRUK)

References:

1. Altman DG, Royston P: What do we mean by validating a prognostic
model? Statistics in Medicine 2000, 19(4):453-473.
2. Wyatt JC, Altman DG: Commentary: Prognostic models: clinically
useful or quickly forgotten? British Medical Journal 1995,
311(7019):1539-1541.
3. Burton A, Altman DG: Missing covariate data within cancer
prognostic studies: a review of current reporting and proposed
guidelines. British Journal of Cancer 2004, 91(1):4-8.
4. Rubin DB: Multiple Imputation for Nonresponse in Surveys New York:
John Wiley and Sons; 2004.
5. Graham JW, Olchowski AE, Gilreath TD: How many imputations
are really needed? Some practical clarifications of multiple
imputation theory. Prevention Science 2007, 8(3):206-213.
6. Kenward MG, Carpenter J: Multiple imputation: current perspectives.
Statistical Methods in Medical Research 2007,
16(3):199-218.
7. van Buuren S, Boshuizen HC, Knook DL: Multiple imputation of
missing blood pressure covariates in survival analysis. Statistics
in Medicine 1999, 18(6):681-694.
8. Li KH, Meng XL, Raghunathan TE, Rubin DB: Significance levels
from repeated p-values with multiply-imputed data. Statistica
Sinica 1991, 1(1):65-92.
9. Schafer JL: Analysis of Incomplete Multivariate Data New York: Chapman
and Hall; 1997.
10. Rubin DB, Schenker N: Multiple imputation in health-care databases:
an overview and some applications. Statistics in Medicine
1991, 10(4):585-598.
11. Rubin DB, Schenker N: Multiple imputation for interval estimation
from simple random samples with ignorable nonresponse.
Journal of the American Statistical Association 1986,
81(394):366-374.
12. Hampel FR, Ronchetti EM, Rousseeuw PJ, Stahel WA: Robust statistics.
The approach based on influence functions New York: John Wiley &
Sons; 1986.
13. Ambler G, Brady AR, Royston P: Simplifying a prognostic model:
a simulation study based on clinical data. Statistics in Medicine
2002, 21(24):3803-3822.
14. Peduzzi P, Concato J, Feinstein AR, Holford TR: Importance of
events per independent variable in proportional hazards
regression analysis. II. Accuracy and precision of regression
estimates. Journal of Clinical Epidemiology 1995, 48(12):1503-1510.
15. Harrell FE: Regression Modeling Strategies with Applications to Linear
Models, Logistic Regression, and Survival Analysis New York: Springer-
Verlag; 2001.
16. Schemper M, Stare J: Explained variation in survival analysis.
Statistics in Medicine 1996, 15(19):1999-2012.
17. Schemper M, Henderson R: Predictive accuracy and explained
variation in Cox regression. Biometrics 2000, 56(1):249-255.
18. O'Quigley J, Xu RH, Stare J: Explained randomness in proportional
hazards models. Statistics in Medicine 2005, 24(3):479-489.
19. Harrell FE, Lee KL, Mark DB: Multivariable prognostic models:
issues in developing models, evaluating assumptions and
adequacy, and measuring and reducing errors. Statistics in
Medicine 1996, 15(4):361-387.
20. Royston P, Sauerbrei W: A new measure of prognostic separation
in survival data. Statistics in Medicine 2004, 23(5):723-748.
21. van Houwelingen HC, Le Cessie S: Predictive value of statistical
models. Statistics in Medicine 1990, 9(1):1303-1325.
22. Meng XL, Rubin DB: Performing likelihood ratio tests with
multiply-imputed data sets. Biometrika 1992, 79(1):103-111.
23. Fisher RA: Statistical Methods for Research Workers Edinburgh: Oliver
and Boyd Ltd; 1941.
24. Hosmer DW, Lemeshow S: Applied survival analysis – Regression modeling
of time to event data New York: John Wiley & Sons; 1999.
25. Collett D: Modelling survival data in medical research Second edition.
London: Chapman & Hall/CRC; 2003.
26. Thomsen BL, Keiding N, Altman DG: A note on the calculation of
expected survival, illustrated by the survival of liver transplant
patients. Statistics in Medicine 1991, 10(5):733-738.
27. Clark TG, Altman DG: Developing a prognostic model in the
presence of missing data. an ovarian cancer case study. Journal
of Clinical Epidemiology 2003, 56(1):28-37.
28. Sinharay S, Stern HS, Russell D: The use of multiple imputation
for the analysis of missing data. Psychological Methods 2001,
6(4):317-329.
29. Gill S, Loprinzi CL, Sargent DJ, Thome SD, Alberts SR, Haller DG,
Benedetti J, Francini G, Shepherd LE, Seitz JF, et al.: Pooled analysis
of fluorouracil-based adjuvant therapy for stage II and III
colon cancer: Who benefits and by how much? Journal of Clinical
Oncology 2004, 22(10):1797-1806.
30. Clark TG, Stewart ME, Altman DG, Gabra H, Smyth JF: A prognostic
model for ovarian cancer. British Journal of Cancer 2001,
85(7):944-952.
31. Rouxel A, Hejblum G, Bernier MO, Boelle PY, Menegaux F, Mansour
G, Hoang C, Aurengo A, Leenhardt L: Prognostic factors associated
with the survival of patients developing loco-regional
recurrences of differentiated thyroid carcinomas. J Clin Endocrinol
Metab 2004, 89(11):5362-5368.
32. Stadler WM, Huo DZ, George C, Yang XM, Ryan CW, Karrison T, Zimmerman
TM, Vogelzang NJ: Prognostic factors for survival with
gemcitabine plus 5-fluorouracil based regimens for metastatic
renal cancer. Journal of Urology 2003, 170(4):1141-1145.
33. Vaughn G, Detels R: Protease inhibitors and cardiovascular disease:
analysis of the Los Angeles County adult spectrum of
disease cohort. AIDS Care 2007, 19(4):492-499.
34. Orsini N, Mantzoros CS, Wolk A: Association of physical activity
with cancer incidence, mortality, and survival: a populationbased
study of men. British Journal of Cancer 2008, 98(11):1864-1869.
35. Mertens AC, Yasui Y, Neglia JP, Potter JD, Nesbit ME, Ruccione K,
Smithson WA, Robison LL: Late mortality experience in fiveyear
survivors of childhood and adolescent cancer: The childhood
cancer survivor study. Journal of Clinical Oncology 2001,
19(13):3163-3172.
36. Serrat C, Gomez G, de Olalla PG, Cayla JA: CD4+ lymphocytes
and tuberculin skin test as survival predictors in pulmonary
tuberculosis HIV-infected patients. International Journal of Epidemiology
1998, 27(4):703-712.
37. Bärnighausen T, Tanser F, Gqwede Z, Mbizana C, Herbst K, Newell
M-L: High HIV incidence in a community with high HIV prevalence
in rural South Africa: findings from a prospective population-
based study. AIDS 2008, 22(1):139-144.
38. Harel O: The estimation of R^2 and adjusted R^2 in incomplete
data sets using multiple imputation. Journal of Applied Statistics
2009 in press. http://www.informaworld.com/10.1080/
02664760802553000
39. Heymans MW, van Buuren S, Knol DL, van Mechelen W, de Vet
HCW: Variable selection under multiple imputation using
the bootstrap in a prognostic study. BMC Medical Research Methodology
2007, 7:33.
40. Hoeting JA, Madigan D, Raftery AE, Volinsky CT: Bayesian model
averaging: A tutorial. Statistical Science 1999, 14(4):382-401.