The Library

Survey of the quality of experimental design, statistical analysis and reporting of research using animals

Tools

Kilkenny, Carol, Parsons, Nicholas R., Kadyszewski, Ed, Festing, Michael F. W., Cuthill, Innes C., Fry, Derek, Hutton, Jane (Statistician) and Altman, Douglas G.. (2009) Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PL o S One, Vol.4 (No.11). Article no. e7824. ISSN 1932-6203

Preview

Text
WRAP_Hutton_journal.pone.0007824.pdf - Draft Version
Download (385Kb)

Official URL: http://dx.doi.org/10.1371/journal.pone.0007824

Abstract

For scientific, ethical and economic reasons, experiments involving animals should be appropriately designed, correctly analysed and transparently reported. This increases the scientific validity of the results, and maximises the knowledge gained from each experiment. A minimum amount of relevant information must be included in scientific publications to ensure that the methods and results of a study can be reviewed, analysed and repeated. Omitting essential information can raise scientific and ethical concerns. We report the findings of a systematic survey of reporting, experimental design and statistical analysis in published biomedical research using laboratory animals. Medline and EMBASE were searched for studies reporting research on live rats, mice and non-human primates carried out in UK and US publicly funded research establishments. Detailed information was collected from 271 publications, about the objective or hypothesis of the study, the number, sex, age and/or weight of animals used, and experimental and statistical methods. Only 59% of the studies stated the hypothesis or objective of the study and the number and characteristics of the animals used. Appropriate and efficient experimental design is a critical component of high-quality science. Most of the papers surveyed did not use randomisation (87%) or blinding (86%), to reduce bias in animal selection and outcome assessment. Only 70% of the publications that used statistical methods described their methods and presented the results with a measure of error or variability. This survey has identified a number of issues that need to be addressed in order to improve experimental design and reporting in publications describing research using animals. Scientific publication is a powerful and important source of information; the authors of scientific publications therefore have a responsibility to describe their methods and results comprehensively, accurately and transparently, and peer reviewers and journal editors share the responsibility to ensure that published studies fulfil these criteria.

Item Type:

Journal Article

Subjects:

Q Science > Q Science (General)

Divisions:

Faculty of Medicine > Warwick Medical School > Health Sciences
Faculty of Science > Statistics
Faculty of Medicine > Warwick Medical School

Library of Congress Subject Headings (LCSH):

Animal experimentation -- Methodology

Journal or Publication Title:

PL o S One

Publisher:

Public Library of Science

ISSN:

1932-6203

Official Date:

30 November 2009

Dates:

Date	Event
30 November 2009	Published

Volume:

Vol.4

Number:

No.11

Number of Pages:

Page Range:

Article no. e7824

Identification Number:

10.1371/journal.pone.0007824

Status:

Peer Reviewed

Publication Status:

Published

Access rights to Published version:

Open Access

Funder:

National Centre for the Replacement, Refinement, and Reduction of Animals in Research (Great Britain) (NC3R), National Institutes of Health (U.S.). Office of Laboratory Animal Welfare (NIH/OLAW)

References:

1. Russell WMS, Burch RL (1959) The Principles of Humane Experimental
Technique. London: Metheun. ;Available: http://altweb.jhsph.edu/pubs/
books/humane_exp/het-toc#. Accessed 20 October 2009.
2. Festing MW, Overend P, Gaines Das R, Cortina Borja M, Berdoy M (2002) The
design of animal experiments: Reducing the use of animals in research through
better experimental design. In: Laboratory Animal Handbooks No 14. London:
Royal Society of Medicine Press Ltd (for Laboratory Animals Ltd). pp 1–16.
3. Festing MF, Altman DG (2002) Guidelines for the design and statistical analysis
of experiments using laboratory animals. ILAR Journal 43(3): 244–258.
4. Johnson PD, Besselsen DG (2002) Practical aspects of experimental design in
animal research. ILAR Journal 43(4): 202–206.
5. Shaw R, Festing MF, Peers I, Furlong L (2002) Use of factorial designs to
optimize animal experiments and reduce animal use. ILAR J 43(4): 223–
232.
6. Freeman J, Walters S, Campbell M (2008) How to Display Data. London:
Blackwell BMJ books.
7. McCance I (1995) Assessment of statistical procedures used in papers in the
Australian Veterinary Journal. Australian Veterinary Journal 72(9): 322–328.
8. Smith JA, Birke L, Sadler D (1997) Reporting animal use in scientific papers.
Laboratory Animals 31: 312–317.
9. O ¨ brink KJ, Rehbinder C (2000) Animal definition: a necessity for the validity of
animal experiments? Laboratory Animals 34: 121–130.
10. Working Committee for the Biological characterisation of laboratory animals/
GV-Solas (1985) Guidelines for specification of animals and husbandry methods
when reporting the results of animal experiments. Laboratory Animals 19:
106–108.
11. Alfaro V (2005) Specification of laboratory animal use in scientific articles:
Current low detail in the journal’s instructions for authors and some proposals.
Methods and Findings in Experimental and Clinical Pharmacology 27(7):
495–502.
12. Dell RB, Holleran S, Ramakrishnan R (2002) Sample size determination. ILAR
Journal 43(4): 207–213.
13. Beynen AC, Festing MFW, van Montford MAJ (2003) Design of animal
experiments. Principles of Laboratory Animal Science, 2nd edn. van
Zutphen LFM, Baumans V, Beynen AC, eds. Amsterdam: Elsevier. pp 219–249.
14. Cochrane AL (1972) The History of the Measurement of III health.
International Journal of Epidemiology 1: 89–92.
15. Bebarta V, Luyten D, Heard K (2003) Emergency medicine animal research:
does use of randomisation and blinding affect the results? Academic Emergency
Medicine 10(6): 684–7.
16. Macleod MR, van der Worp HB, Sena ES, Howells DW, Dirnagl U, et al.
(2008) Evidence for the efficacy of NXY -059 in Experimental Focal Cerebral
Ischemia is confounded by study quality. Stroke 39: 2824–2829.
17. Nakagawa S, Cuthill IC (2007) Effect size, confidence interval and statistical
significance: A practical guide for biologists. Biological Reviews 82: 591–605.
18. Note: These surveys have been chosen based on quality and relevance criteria; a
systematic search for all surveys of this type was not performed.
19. Macleod MR, Fisher M, O’Collins V, et al. (2009) Good laboratory practice.
Preventing introduction of bias at the bench. Stroke 40: 50–52.
20. Tooth L, Ware R, Bain C, Purdie DM, Dobson A (2005) Quality of reporting of
observational longitudinal research. American Journal of Epidemiology 161:
280–288.
21. Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Reitsma JB, et al. (2005)
Quality of reporting of diagnostic accuracy studies. Radiology 235: 347–353.
22. Jafari P, Azuaje F (2006) An assessment of recently published gene expression
analyses: reporting experimental design and statistics. BMC Medical Informatics
and Decision Making 6: 27.
23. Morris CE, Bardin M, Berge O, Frey-Klett P, Fromin N (2002) Microbial
biodiversity: Approaches to experimental design and hypothesis testing in
primary scientific literature from 1975 to 1999. Microbiology and Molecular
Biology Reviews 66(4): 592–616.
24. Bennett LT, Adams M A (2004) Assessment of ecological effects due to forest
harvesting: approaches and statistical issues. Journal of Applied Ecology 41:
585–598.
25. Hainsworth AH, Markus HS (2008) Do in vivo experimental models reflect
human cerebral small vessel disease? A systematic review. Journal of Cerebral
Blood Flow and metabolism 28: 1877–1891.
26. Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, et al. (2001) The
Revised CONSORT statement for reporting randomized trials: explanation and
elaboration. Annals of Internal Medicine 134: 663–694.
27. Altman DG (2005) Endorsement of the CONSORT statement by high impact
medical journals: survey of instructions for authors. British Medical Journal 330:
1056–1057.
28. Goodacre R, Broadhurst D, Smilde AK, Kristal BS, Baker DJ, et al. (2007)
Proposed minimum reporting standards for data analysis in metabolomics.
Metabolomics 3: 231–241.
29. Altman DG, Simera I, Hoey J, Moher D, Schulz K (2008) EQUATOR:
reporting guidelines for health research. Lancet 371: 1149–1150.
30. www.equator-network.org.
31. Peters JL, Sutton AJ, Jones DR, Rushton L, Abrams KR (2006) A systematic
review of systematic reviews and meta-analyses of animal experiments with
guidelines for reporting. Journal of Environmental Science and Health Part BPesticides
Food Contaminants and Agricultural Wastes 41: 1245–1258.
32. Stone SP, Cooper BS, Kibbler CC, Cookson BD, Roberts JA (2007) The
ORION statement: guidelines for transparent reporting of outbreak reports and
intervention studies of nosocomial infection. Journal of Antimicrobial Chemotherapy
59: 833–840.
33. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC (2007) The
strengthening the reporting of observational studies in epidemiology (STROBE)
statement: Guidelines for reporting observational studies. Lancet 370:
1453–1457.