The Library

A realistic evaluation : the case of protocol-based care

Tools

Rycroft-Malone, Jo, Fontenla, Maria, Bick, Debra and Seers, Kate. (2010) A realistic evaluation : the case of protocol-based care. Implementation Science, Vol.5 . Article 38. ISSN 1748-5908

PDF
WRAP_Seers_Realistic_evaluation.pdf - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (858Kb)

Official URL: http://dx.doi.org/10.1186/1748-5908-5-38

Abstract

Background 'Protocol based care' was envisioned by policy makers as a mechanism for delivering on the service improvement agenda in England. Realistic evaluation is an increasingly popular approach, but few published examples exist, particularly in implementation research. To fill this gap, within this paper we describe the application of a realistic evaluation approach to the study of protocol-based care, whilst sharing findings of relevance about standardising care through the use of protocols, guidelines, and pathways. Methods Situated between positivism and relativism, realistic evaluation is concerned with the identification of underlying causal mechanisms, how they work, and under what conditions. Fundamentally it focuses attention on finding out what works, for whom, how, and in what circumstances. Results In this research, we were interested in understanding the relationships between the type and nature of particular approaches to protocol-based care (mechanisms), within different clinical settings (context), and what impacts this resulted in (outcomes). An evidence review using the principles of realist synthesis resulted in a number of propositions, i.e., context, mechanism, and outcome threads (CMOs). These propositions were then 'tested' through multiple case studies, using multiple methods including non-participant observation, interviews, and document analysis through an iterative analysis process. The initial propositions (conjectured CMOs) only partially corresponded to the findings that emerged during analysis. From the iterative analysis process of scrutinising mechanisms, context, and outcomes we were able to draw out some theoretically generalisable features about what works, for whom, how, and what circumstances in relation to the use of standardised care approaches (refined CMOs). Conclusions As one of the first studies to apply realistic evaluation in implementation research, it was a good fit, particularly given the growing emphasis on understanding how context influences evidence-based practice. The strengths and limitations of the approach are considered, including how to operationalise it and some of the challenges. This approach provided a useful interpretive framework with which to make sense of the multiple factors that were simultaneously at play and being observed through various data sources, and for developing explanatory theory about using standardised care approaches in practice.

Item Type:	Journal Article
Subjects:	R Medicine > R Medicine (General)
Divisions:	Faculty of Social Sciences > School of Health and Social Studies
Library of Congress Subject Headings (LCSH):	Medical care, Medical protocols -- Evaluation
Journal or Publication Title:	Implementation Science
Publisher:	BioMed Central Ltd.
ISSN:	1748-5908
Date:	26 May 2010
Volume:	Vol.5
Page Range:	Article 38
Identification Number:	10.1186/1748-5908-5-38
Status:	Peer Reviewed
Access rights to Published version:	Open Access
Funder:	National Institute for Health Research (Great Britain) (NIHR)
Grant number:	SDO/78/2004 (NIHR)
References:	1. Department of Health: The New NHS. Modern. Dependable 1997 [http:// www.dh.gov.uk/en/Publicationsandstatistics/Publications/ PublicationsPolicyAndGuidance/DH_4008869]. London: The Stationary Office (accessed 13 May 2010) 2. Department of Health The NHS Plan: A Plan for Investment. A Plan for Reform 2000 [http://www.dh.gov.uk/prod_consum_dh/groups/ dh_digitalassets/@dh/@en/documents/digitalasset/dh_4055783.pdf]. London: The Stationary Office (accessed 8 December 2009) 3. Department of Health: Making a Difference: strengthening the nursing, midwifery and health visiting contribution to health and healthcare 1999 [http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/ @en/documents/digitalasset/dh_4074704.pdf]. The Stationary Office, London (accessed 8 December 2009) 4. NHS Modernisation Agency & National Institute for Clinical Excellence: Protocol-based care underpinning improvement - what is protocol-based care? 2002 [http://www.institute.nhs.uk/ quality_and_service_improvement_tools/ quality_and_service_improvement_tools/protocol_based_care.html]. (accessed 13 May 2010) 5. Ilott I, Rick J, Patterson M, Turgoose C, Lacey A: What is protocol-based care? A concept analysis. Journal of Nursing Management 2006, 14:544-552. 6. Rycroft-Malone J, Morrell C, Bick D: Protocol-based care: The research agenda. Nursing Standard 2004, 19(6):33-36. 7. Hunter B, Segrott J: Re-mapping client journeys and professional identities: A review of the literature on clinical pathways. International Journal of Nursing Studies 2008, 45(4):608-625. 8. Rycroft-Malone J, Fontenla M, Bick D, Seers K: Protocol-based care: Impact on roles and service delivery. Journal of Evaluation in Clinical Practice 2008, 14:867-873. 9. Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K, Titchen A, Estabrooks C: Ingredients for Change: Revisiting a conceptual model. Quality & Safety in Health Care 2002, 11:174-180. 10. Rycroft-Malone J, Harvey G, Seers K, Kitson A, McCormack B, Titchen A: An exploration of the factors that influence the implementation of evidence into practice. Journal of Clinical Nursing 2004, 13:913-924. 11. Dopson S, Fizgerald L: Knowledge to Action Oxford: Oxford University Press; 2005. 12. Greenhalgh T, Robert G, McFarlane F, Bate P, Kyriakidou O: Diffusion of Innovations in Service Organisations: Systematic Review and Recommendations. The Milbank Quarterly 2004, 82(4):581-629. 13. Stetler C, Ritchie J, Rycroft-Malone J, Shultz A, Charns M: Institutionalizing evidence-based practice: an organizational case study using a model of strategic change. Implementation Science 2009, 4:78. 14. Buchan H, Lourey E, D'Este C, Sanson-Fisher R: Effectiveness of strategies to encourage general practitioners to accept an offer of free access to online evidence-based information: a randomised controlled trial. Implementation Science 2009, 4:68. 15. Dobbins M, Hanna SE, Ciliska D, Manske S, Cameron R, Mercer SL, O'Mara L, DeCorby K, Robeson P: A randomized controlled trial evaluating the impact of knowledge translation and exchange strategies. Implementation Science 2009, 4:61. 16. European Commission Working Time Directive: [http://ec.europa.eu/ social/main.jsp?catId=706&langId=en&intPageId=205]. (accessed 9 December 2009) 17. Redfern S, Christian S, Norman I: Evaluating change in health care practice: lessons from three studies. Journal of Evaluation in Clinical Practice 2003, 9(2):239-250. 18. Byng R, Norman I, Redfern S: Using realistic evaluation to evaluate a practice level intervention to improve primary healthcare for patients with long term mental illness. Evaluation 2005, 11(1):69-93. 19. Tolson D, McIntosh J, Loftus L, Cormie P: Developing a managed clinical network in palliative care: a realistic evaluation. International Journal of Nursing Studies 2007, 44(2):183-195. 20. Greenhalgh T, Humphrey C, Hughes J, MacFarlane F, Butler C, Pawson R: How do you modernize a health service? A realist evaluation of whole scale transformation in London. Milbank Quarterly 2009, 87(2):391-416. 2009 21. Pawson R, Tilley N: Realistic evaluation London: Sage; 1997. 22. Delanty G: Social Science: Beyond Constructivism and Realism Open University Press, Buckingham; 1997. 23. McEvoy P, Richards D: Critical realism: A way forward for evaluation research in nursing? Journal of Advanced Nursing 2003, 43:411-420. 24. Maxwell JA: Using qualitative methods for causal explanation. Field Methods 2004, 16(3):243-264. 25. Wilson V, McCormack B: Critical Realism as emancipatory action: the case for realistic evaluation in practice development. Nursing Philosophy 2006, 7:45-57. 26. Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist review - a new method of systematic review designed for complex interventions. Journal of Health Service Research & Policy 2005, 10(1 Suppl 1):21-34. 27. Pawson R: Evidence-Based Policy: a Realist Perspective London: Sage; 2006. 28. Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist synthesis: an introduction. ESRC Research Methods Programme RMP Methods Paper 2/ 2004 [http://www.ccsr.ac.uk/methods/publications/#Papers]. (accessed 8 December 2009) 29. Rycroft-Malone J, Fontenla M, Bick D, Seers K: Protocol-Based Care Evaluation Project. Final Report for the National Institute for Health Research Service Delivery and Organisation Programme 2008 [http:// www.sdo.nihr.ac.uk/projdetails.php?ref=08-1405-078]. (accessed 13 May 2010) 30. de Luc K: Care pathways: an evaluation of their effectiveness. Journal of Advanced Nursing 2000, 32(2):485-496. 31. Manias E, Street A: Legitimating of Nurses' Knowledge Though Policies and Protocols in Clinical Practice. Journal of Advanced Nursing 2000, 4(32):1467-75. 32. Yin RK: Applications of case study research Thousand Oaks, CA: Sage; 1993. 33. Yin RK: Case study research. Designs and methods 2nd edition. Thousand Oaks CA: Sage; 1994. 34. Lincoln YS, Guba EC: Sage, Newbury Park; 1985. 35. Dopson S, Fitzgerald L, Ferlie E, Gabbay J, Locock L: No magic targets! Changing clinical practice to become more evidence based. Health Care Management Review 2002, 27(3):35-47. 36. Rycroft-Malone J, Dopson S, Degner L, Hutchinson AM, Morgan D, Stewart N, Estabrooks CA: Study protocol for the translating research in elder care (TREC): building context through case studies in long-term care project (project 2). Implementation Science 2009, 4:53. (11 August 2009) 37. Rycroft-Malone J: Evidence-informed practice: From individual to context. Journal of Nursing Management 2008, 16(4):404-408. 38. McCormack B, Kitson A, Harvey G, Rycroft-Malone J, Seers K, Titchen A: Getting Evidence into Practice: The meaning of 'context'. Journal of Advanced Nursing 2002, 38(1):94-104. 39. Scott SD, Estabrooks CA, Allen M, Pollock C: A Context of Uncertainty: How context shapes nurses' research utilization behaviours. Qualitative Health Research 2008, 18(3):347-357. 40. ICEBeRG Group: Designing theoretically-informed implementation interventions. Implementation Science 2006 2006, 1:4. 41. Rycroft-Malone J: Theory and Knowledge Translation: Setting some coordinates. Nursing Research 2007, 56(Suppl 4):S78-S85. 42. Rycroft-Malone J, Bucknall T: Models and Frameworks for Implementing Evidence-Based Practice: Linking Evidence to Action Oxford: Wiley-Blackwell; 2010.
URI:	http://wrap.warwick.ac.uk/id/eprint/3293