The Library

Survey of the quality of experimental design, statistical analysis and reporting of research using animals

Tools

Kilkenny, Carol, Parsons, Nicholas R., Kadyszewski, Ed, Festing, Michael F. W., Cuthill, Innes C., Fry, Derek, Hutton, Jane L. and Altman, Douglas G.. (2009) Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PL o S One, Vol.4 (No.11). Article no. e7824. ISSN 1932-6203

Preview

Text
WRAP_Hutton_journal.pone.0007824.pdf - Draft Version
Download (385Kb)

Official URL: http://dx.doi.org/10.1371/journal.pone.0007824

Abstract

For scientific, ethical and economic reasons, experiments involving animals should be appropriately designed, correctly analysed and transparently reported. This increases the scientific validity of the results, and maximises the knowledge gained from each experiment. A minimum amount of relevant information must be included in scientific publications to ensure that the methods and results of a study can be reviewed, analysed and repeated. Omitting essential information can raise scientific and ethical concerns. We report the findings of a systematic survey of reporting, experimental design and statistical analysis in published biomedical research using laboratory animals. Medline and EMBASE were searched for studies reporting research on live rats, mice and non-human primates carried out in UK and US publicly funded research establishments. Detailed information was collected from 271 publications, about the objective or hypothesis of the study, the number, sex, age and/or weight of animals used, and experimental and statistical methods. Only 59% of the studies stated the hypothesis or objective of the study and the number and characteristics of the animals used. Appropriate and efficient experimental design is a critical component of high-quality science. Most of the papers surveyed did not use randomisation (87%) or blinding (86%), to reduce bias in animal selection and outcome assessment. Only 70% of the publications that used statistical methods described their methods and presented the results with a measure of error or variability. This survey has identified a number of issues that need to be addressed in order to improve experimental design and reporting in publications describing research using animals. Scientific publication is a powerful and important source of information; the authors of scientific publications therefore have a responsibility to describe their methods and results comprehensively, accurately and transparently, and peer reviewers and journal editors share the responsibility to ensure that published studies fulfil these criteria.

Item Type:	Journal Article
Subjects:	Q Science > Q Science (General)
Divisions:	Faculty of Medicine > Warwick Medical School > Health Sciences Faculty of Science > Statistics Faculty of Medicine > Warwick Medical School
Library of Congress Subject Headings (LCSH):	Animal experimentation -- Methodology
Journal or Publication Title:	PL o S One
Publisher:	Public Library of Science
ISSN:	1932-6203
Date:	30 November 2009
Volume:	Vol.4
Number:	No.11
Number of Pages:	11
Page Range:	Article no. e7824
Identification Number:	10.1371/journal.pone.0007824
Status:	Peer Reviewed
Publication Status:	Published
Access rights to Published version:	Open Access
Funder:	National Centre for the Replacement, Refinement, and Reduction of Animals in Research (Great Britain) (NC3R), National Institutes of Health (U.S.). Office of Laboratory Animal Welfare (NIH/OLAW)
References:	1. Russell WMS, Burch RL (1959) The Principles of Humane Experimental Technique. London: Metheun. ;Available: http://altweb.jhsph.edu/pubs/ books/humane_exp/het-toc#. Accessed 20 October 2009. 2. Festing MW, Overend P, Gaines Das R, Cortina Borja M, Berdoy M (2002) The design of animal experiments: Reducing the use of animals in research through better experimental design. In: Laboratory Animal Handbooks No 14. London: Royal Society of Medicine Press Ltd (for Laboratory Animals Ltd). pp 1–16. 3. Festing MF, Altman DG (2002) Guidelines for the design and statistical analysis of experiments using laboratory animals. ILAR Journal 43(3): 244–258. 4. Johnson PD, Besselsen DG (2002) Practical aspects of experimental design in animal research. ILAR Journal 43(4): 202–206. 5. Shaw R, Festing MF, Peers I, Furlong L (2002) Use of factorial designs to optimize animal experiments and reduce animal use. ILAR J 43(4): 223– 232. 6. Freeman J, Walters S, Campbell M (2008) How to Display Data. London: Blackwell BMJ books. 7. McCance I (1995) Assessment of statistical procedures used in papers in the Australian Veterinary Journal. Australian Veterinary Journal 72(9): 322–328. 8. Smith JA, Birke L, Sadler D (1997) Reporting animal use in scientific papers. Laboratory Animals 31: 312–317. 9. O ¨ brink KJ, Rehbinder C (2000) Animal definition: a necessity for the validity of animal experiments? Laboratory Animals 34: 121–130. 10. Working Committee for the Biological characterisation of laboratory animals/ GV-Solas (1985) Guidelines for specification of animals and husbandry methods when reporting the results of animal experiments. Laboratory Animals 19: 106–108. 11. Alfaro V (2005) Specification of laboratory animal use in scientific articles: Current low detail in the journal’s instructions for authors and some proposals. Methods and Findings in Experimental and Clinical Pharmacology 27(7): 495–502. 12. Dell RB, Holleran S, Ramakrishnan R (2002) Sample size determination. ILAR Journal 43(4): 207–213. 13. Beynen AC, Festing MFW, van Montford MAJ (2003) Design of animal experiments. Principles of Laboratory Animal Science, 2nd edn. van Zutphen LFM, Baumans V, Beynen AC, eds. Amsterdam: Elsevier. pp 219–249. 14. Cochrane AL (1972) The History of the Measurement of III health. International Journal of Epidemiology 1: 89–92. 15. Bebarta V, Luyten D, Heard K (2003) Emergency medicine animal research: does use of randomisation and blinding affect the results? Academic Emergency Medicine 10(6): 684–7. 16. Macleod MR, van der Worp HB, Sena ES, Howells DW, Dirnagl U, et al. (2008) Evidence for the efficacy of NXY -059 in Experimental Focal Cerebral Ischemia is confounded by study quality. Stroke 39: 2824–2829. 17. Nakagawa S, Cuthill IC (2007) Effect size, confidence interval and statistical significance: A practical guide for biologists. Biological Reviews 82: 591–605. 18. Note: These surveys have been chosen based on quality and relevance criteria; a systematic search for all surveys of this type was not performed. 19. Macleod MR, Fisher M, O’Collins V, et al. (2009) Good laboratory practice. Preventing introduction of bias at the bench. Stroke 40: 50–52. 20. Tooth L, Ware R, Bain C, Purdie DM, Dobson A (2005) Quality of reporting of observational longitudinal research. American Journal of Epidemiology 161: 280–288. 21. Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Reitsma JB, et al. (2005) Quality of reporting of diagnostic accuracy studies. Radiology 235: 347–353. 22. Jafari P, Azuaje F (2006) An assessment of recently published gene expression analyses: reporting experimental design and statistics. BMC Medical Informatics and Decision Making 6: 27. 23. Morris CE, Bardin M, Berge O, Frey-Klett P, Fromin N (2002) Microbial biodiversity: Approaches to experimental design and hypothesis testing in primary scientific literature from 1975 to 1999. Microbiology and Molecular Biology Reviews 66(4): 592–616. 24. Bennett LT, Adams M A (2004) Assessment of ecological effects due to forest harvesting: approaches and statistical issues. Journal of Applied Ecology 41: 585–598. 25. Hainsworth AH, Markus HS (2008) Do in vivo experimental models reflect human cerebral small vessel disease? A systematic review. Journal of Cerebral Blood Flow and metabolism 28: 1877–1891. 26. Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, et al. (2001) The Revised CONSORT statement for reporting randomized trials: explanation and elaboration. Annals of Internal Medicine 134: 663–694. 27. Altman DG (2005) Endorsement of the CONSORT statement by high impact medical journals: survey of instructions for authors. British Medical Journal 330: 1056–1057. 28. Goodacre R, Broadhurst D, Smilde AK, Kristal BS, Baker DJ, et al. (2007) Proposed minimum reporting standards for data analysis in metabolomics. Metabolomics 3: 231–241. 29. Altman DG, Simera I, Hoey J, Moher D, Schulz K (2008) EQUATOR: reporting guidelines for health research. Lancet 371: 1149–1150. 30. www.equator-network.org. 31. Peters JL, Sutton AJ, Jones DR, Rushton L, Abrams KR (2006) A systematic review of systematic reviews and meta-analyses of animal experiments with guidelines for reporting. Journal of Environmental Science and Health Part BPesticides Food Contaminants and Agricultural Wastes 41: 1245–1258. 32. Stone SP, Cooper BS, Kibbler CC, Cookson BD, Roberts JA (2007) The ORION statement: guidelines for transparent reporting of outbreak reports and intervention studies of nosocomial infection. Journal of Antimicrobial Chemotherapy 59: 833–840. 33. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC (2007) The strengthening the reporting of observational studies in epidemiology (STROBE) statement: Guidelines for reporting observational studies. Lancet 370: 1453–1457.
URI:	http://wrap.warwick.ac.uk/id/eprint/16760