The Library
Double-counting of populations in evidence synthesis in public health : a call for awareness and future methodological development
Tools
Hussein, Humaira, Nevill, Clareece R., Meffen, Anna, Abrams, Keith R., Bujkiewicz, Sylwia, Sutton, Alex J. and Gray, Laura J. (2022) Double-counting of populations in evidence synthesis in public health : a call for awareness and future methodological development. BMC Public Health, 22 (1). 1827. doi:10.1186/s12889-022-14213-6 ISSN 1471-2458.
|
PDF
WRAP-Double-counting-of-populations-in-evidence-synthesis-in-public-health-a-call-for-awareness-and-future-methodological-development-Abrams-2022.pdf - Published Version - Requires a PDF viewer. Available under License Creative Commons Attribution 4.0. Download (1626Kb) | Preview |
Official URL: http://dx.doi.org/10.1186/s12889-022-14213-6
Abstract
Background
There is a growing interest in the inclusion of real-world and observational studies in evidence synthesis such as meta-analysis and network meta-analysis in public health. While this approach offers great epidemiological opportunities, use of such studies often introduce a significant issue of double-counting of participants and databases in a single analysis. Therefore, this study aims to introduce and illustrate the nuances of double-counting of individuals in evidence synthesis including real-world and observational data with a focus on public health.
Methods
The issues associated with double-counting of individuals in evidence synthesis are highlighted with a number of case studies. Further, double-counting of information in varying scenarios is discussed with potential solutions highlighted.
Results
Use of studies of real-world data and/or established cohort studies, for example studies evaluating the effectiveness of therapies using health record data, often introduce a significant issue of double-counting of individuals and databases. This refers to the inclusion of the same individuals multiple times in a single analysis. Double-counting can occur in a number of manners, such as, when multiple studies utilise the same database, when there is overlapping timeframes of analysis or common treatment arms across studies. Some common practices to address this include synthesis of data only from peer-reviewed studies, utilising the study that provides the greatest information (e.g. largest, newest, greater outcomes reported) or analysing outcomes at different time points.
Conclusions
While common practices currently used can mitigate some of the impact of double-counting of participants in evidence synthesis including real-world and observational studies, there is a clear need for methodological and guideline development to address this increasingly significant issue.
Peer Review reports
Introduction
Both in the evaluation of health technologies and epidemiological research, systematic reviews and meta-analysis are regarded as providing high quality evidence [1, 2]. With the heightening interest in studies reporting the use of real-world data in health research literature, which include observational studies using registry or electronic health record data collected routinely in clinical practice, the incorporation of these studies in evidence synthesis is becoming increasingly common [3,4,5,6]. Utilising data from all available sources, including observational studies, can provide many benefits in epidemiology, such as increased power and more generalizable results. However, this can often introduce a number of analytical problems such as confounding, significant heterogeneity and misclassification bias within the non-randomised evidence.
While methods such as meta-regression have been considered to address these issues [7, 8], a significant problem that has received little attention within public health research is the double-counting, also referred to as sample overlap, of individuals and databases when including such studies in evidence syntheses. With the increased use of cohort and real-world data in evidence synthesis, double-counting has the potential to become a significant issue. Some aspects of double-counting have been discussed by Senn (2009) and Lunny et al.,(2021), specifically in the context of whole studies or study arms which were being included multiple times in the meta-analysis [9, 10]. More attention has been given to this issue in the fields of social science, education, economics and finance, where analytical approaches to dealing with such issues have been suggested [11, 12]. However, currently there is no published guidance available on how to address this.
Sample overlap between studies will lead to spuriously high precision in meta-analysis and is also potentially a source of bias. Due to this, many reviewers choose to exclude or adjust for studies where there is an overlap of participants. It may not be obvious if studies contain overlapping patients and so double-counting of individuals in a synthesis may exist without the reviewer’s knowledge. Whilst guidance documents for conducting systematic reviews and meta-analysis of intervention and prevalence/incidence studies exist, none of these consider the effect of the large magnitude of sample overlap expected in whole population studies on meta-analysis results [13, 14]. Therefore, this paper aims to highlight and illustrate some of the specific methodological and practical aspects of double-counting of individuals and datasets that should be considered in evidence syntheses that include real-world and observational data using a number of public health case studies.
Item Type: | Journal Article | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Subjects: | R Medicine > R Medicine (General) R Medicine > RA Public aspects of medicine |
|||||||||||||||
Divisions: | Faculty of Science, Engineering and Medicine > Science > Statistics | |||||||||||||||
Library of Congress Subject Headings (LCSH): | Meta-analysis, Public health -- Research, Medical informatics, Epidemiology -- Research, Big data | |||||||||||||||
Journal or Publication Title: | BMC Public Health | |||||||||||||||
Publisher: | BioMed Central Ltd. | |||||||||||||||
ISSN: | 1471-2458 | |||||||||||||||
Official Date: | 27 September 2022 | |||||||||||||||
Dates: |
|
|||||||||||||||
Volume: | 22 | |||||||||||||||
Number: | 1 | |||||||||||||||
Number of Pages: | 10 | |||||||||||||||
Article Number: | 1827 | |||||||||||||||
DOI: | 10.1186/s12889-022-14213-6 | |||||||||||||||
Status: | Peer Reviewed | |||||||||||||||
Publication Status: | Published | |||||||||||||||
Access rights to Published version: | Open Access (Creative Commons) | |||||||||||||||
Date of first compliant deposit: | 17 October 2022 | |||||||||||||||
Date of first compliant Open Access: | 18 October 2022 | |||||||||||||||
RIOXX Funder/Project Grant: |
|
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year