Royle, Pamela and Waugh, Norman (2003) Literature searching for clinical and cost-effectiveness studies used in health technology assessment reports carried out for the National Institute for Clinical Excellence appraisal system. Health Technology Assessment, Vol.7 (No.34). ISSN 1366-5278.

Preview	Text WRAP_Waugh_lit searchTAR.pdf - Published Version Download (271Kb) | Preview
Preview	Text (Cover sheet) WRAP_waugh8_Coversheet_FinalVersion.pdf Download (168Kb) | Preview

Request Changes to record.

Abstract

Background

In the UK, one part of the remit of the National Institute for Clinical Excellence (NICE) is to carry out a programme of technology appraisals. These are done to a fairly tight timetable in order not to delay the guidance on new technologies. Each appraisal is underpinned by a Technology Assessment Report (TAR) commissioned from a group of academic units.

As the TAR process is relatively new, and is still evolving, the methods used for its literature searching have been largely based on the well-established and documented methods used for Cochrane reviews. These involve comprehensive searching of a variety of sources to protect against bias, but can add substantially to the time and costs of carrying out a review.

However, resource constraints require that TARs are produced as efficiently as possible, and to a tight timetable, which means that not all of the Cochrane methods can be applied, or are appropriate. In addition, it is not known whether the marginal benefits of exhaustive searching justify the costs. The challenge for those undertaking TARs is to know how best to adapt and optimise, and extend when necessary, the Cochrane-based search strategies, so that searching can be done both rapidly and systematically.

Objective

To contribute to making searching for TARs more cost-effective by suggesting an optimum literature retrieval strategy, based on empirical data obtained from a sample of recent TARs, which balances comprehensiveness and efficiency.

Methods

A sample of 20 recent TARs was studied. All sources used to search for clinical and cost-effectiveness studies were recorded. In addition, all studies that were included in the clinical and cost-effectiveness sections of the TARs were identified, and their characteristics recorded, including author, journal, year, study design, study size and quality score. Each was also classified by publication type, and then checked to see whether it was indexed in the following databases: MEDLINE, EMBASE, and then either the Cochrane Controlled Trials Register (CCTR) for clinical effectiveness studies or the NHS Economic Evaluation Database (NHS EED) for the cost-effectiveness studies. Any study not found in at least one of these databases was checked to see whether it was indexed in the Science Citation Index (SCI) and BIOSIS, and the American Society of Clinical Oncology (ASCO) Online if a cancer review. Any studies still not found were investigated further to see whether they were in a number of additional databases.

Results

Sources searched

The median number of sources searched per TAR was 20, and the range was from 13 to 33 sources. Six sources (CCTR, DARE, EMBASE, MEDLINE, NHS EED and sponsor/industry submissions to NICE) were used in all reviews.

Clinical effectiveness studies

There were 424 studies in total. The publication types were: published 80%, meeting abstracts 11.3% and unpublished 8.7%. Eighty per cent of reviews included at least one abstract or unpublished study (60% included at least one abstract and 50% included at least one unpublished study). The median number of studies included per TAR was 19.5 (range 2–41). The median number of participants included per TAR was 2787 (range 69–97,570). Evidence from non-randomised controlled trial (RCT) studies was used in 45% of TARs. The proportion of studies classified either as published in full or as abstracts, and found indexed in the following databases, was: MEDLINE 82.7%, EMBASE 78.6% and CCTR 50.1%. The cumulative percentage of studies found after searching these three databases was 87.3%. Adding SCI, BIOSIS and ASCO Online increased this to 98.2%. Eighty-seven per cent of studies were indexed in both MEDLINE and EMBASE.

Cost-effectiveness studies

The 130 studies were classified as: published 73.1%, unpublished 23.8%, abstracts 1.5% and grey literature 1.5%. The median number of studies used was 4.0. The percentage of studies classified as either published in full or as abstracts, and found indexed in the following databases, was: MEDLINE 86.6%, EMBASE 86.6% and NHS EED 40.2%. The cumulative percentage of these studies found indexed after searching the three databases was 94.8%. Adding SCI and ASCO Online increased this to 97.9%.

Studies used in the economic modelling

The 121 articles were classified as: published 50.4%, abstracts 5.0%, reference sources 17.4%, unpublished 17.4% and grey literature 9.8%. The median number of studies used for the 14 TARs that included an economic model was 9.0 per TAR.

Search terms for identifying non-RCTs

A sensitive search filter, constructed for MEDLINE and using the search terms from the bibliographic records in the included studies, retrieved only 85% of the known sample. Therefore, it is recommended that when searching for non-RCT studies a search is done for the intervention alone, and records are then scanned manually for those that look relevant.

Conclusions

Searching additional databases beyond the Cochrane Library (which includes CCTR, NHS EED and the HTA database), MEDLINE, EMBASE and SCI, plus BIOSIS limited to meeting abstracts only, is seldom effective in retrieving additional studies for inclusion in the clinical and cost-effectiveness sections of TARs (apart from reviews of cancer therapies, where a search of the ASCO database is recommended). A more selective approach to database searching would suffice in most cases and would save resources, thereby making the TAR process more efficient. However, searching non-database sources (including submissions from manufacturers, recent meeting abstracts, contact with experts and checking reference lists) does appear to be a productive way of identifying further studies.

Item Type:	Journal Article
Subjects:	R Medicine > R Medicine (General)
Divisions:	Faculty of Science, Engineering and Medicine > Medicine > Warwick Medical School > Health Sciences > Population, Evidence & Technologies (PET) > Warwick Evidence Faculty of Science, Engineering and Medicine > Medicine > Warwick Medical School > Health Sciences Faculty of Science, Engineering and Medicine > Medicine > Warwick Medical School
Library of Congress Subject Headings (LCSH):	Information storage and retrieval systems -- Medicine, Medicine -- Research, Systematic reviews (Medical research)
Journal or Publication Title:	Health Technology Assessment
Publisher:	National Coordinating Centre for Health Technology Assessment
ISSN:	1366-5278
Official Date:	2003
Dates:	Date Event 2003 Published
Volume:	Vol.7
Number:	No.34
Number of Pages:	64
Status:	Peer Reviewed
Publication Status:	Published
Date of first compliant deposit:	23 December 2015
Date of first compliant Open Access:	23 December 2015
Funder:	NIHR Health Technology Assessment Programme (Great Britain)

Request changes or add full text files to a record

Repository staff actions (login required)

View Item

Downloads