Topps, D., Evans, Rebecca, Thistlethwaite, Jill, Nan Tie, Rod and Ellaway, Rachel. (2009) The one minute mentor : a pilot study assessing medical students’ and residents’ professional behaviours through recordings of clinical preceptors’ immediate feedback. Education for Health, Vol.22 (No.1).

Preview

PDF WRAP_Thistlewaite_one_minute_Mentor.pdf - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader Download (349Kb)

Abstract

Introduction: The assessment of professional development and behaviour is an important issue in the training of medical students and physicians. Several methods have been developed for doing so. What is still needed is a method that combines assessment of actual behaviour in the workplace with timely feedback to learners. Goal: We describe the development, piloting and evaluation of a method for assessing professional behaviour using digital audio recordings of clinical supervisors’ brief feedback. We evaluate the inter-rater reliability, acceptability and feasibility of this approach. Methods: Six medical students in Year 5 and three GP registrars (residents) took part in this pilot project. Each had a personal digital assistant (PDA) and approached their clinical supervisors to give approximately one minute of verbal feedback on professionalism-related behaviours they had observed in the registrar’s clinical encounters. The comments, both in transcribed text format and audio, were scored by five evaluators for competence (the learner’s performance) and confidence (how confident the evaluator was that the comment clearly described an observed behaviour or attribute that was relevant). Students and evaluators were surveyed for feedback on the process. Results: Study evaluators rated 29 comments from supervisors in text and audio format. There was good inter-rater reliability (Cronbach α around 0.8) on competence scores. There was good agreement (paired t-test) between scores across supervisors for assessments of comments in both written and audio formats. Students found the method helpful in providing feedback on professionalism. Evaluators liked having a relatively objective approach for judging behaviours and attributes but found scoring audio comments to be time-consuming. Discussion: This method of assessing learners’ professional behaviour shows potential for providing both formative and summative assessment in a way that is feasible and acceptable to students and evaluators. Initial data shows good reliability but to be valid, training of clinical supervisors is necessary to help them provide useful comments based on defined behaviours and attributes of students. In addition, the validity of the scoring method remains to be confirmed.

Item Type:	Journal Article
Subjects:	L Education > LB Theory and practice of education R Medicine > RA Public aspects of medicine
Divisions:	Faculty of Medicine > Warwick Medical School > Institute of Clinical Education (ICE) Faculty of Medicine > Warwick Medical School
Library of Congress Subject Headings (LCSH):	Medical students -- Rating of, Professional education, Medicine -- Study and teaching -- Audio-visual aids
Journal or Publication Title:	Education for Health
Publisher:	Maastricht University
Date:	May 2009
Volume:	Vol.22
Number:	No.1
Status:	Peer Reviewed
Access rights to Published version:	Open Access
References:	Albanese, M. A. (2000). Challenges in using rater judgements in medical education. Journal Evaluation Clinical Practice, 6(3), 305-319. Baker, R. (2005). Can poorly performing doctors blame their assessment tools? British Medical Journal, 330(7502), 1254. Cruess, S. R., & Cruess, R. L. (1997). Professionalism must be taught. British Medical Journal, 315(7123), 1674-1677. Ginsburg, S., Regehr, G., & Lingard, L. (2003). To be and not to be: the paradox of the emerging professional stance. Medical Education, 37(4), 350-357. Department of Trade and Industry. (2005). 360 degree feedback. Best practice guidelines., from www.dti.gov.uk/bestpractice. Johnson, T. (1972). Professions and Power. London: Macmillan Press Limited. Kilminster, S. M., & Jolly, B. C. (2000). Effective supervision in clinical practice settings: a literature review. Medical Education, 34(10), 827-840. Klein, E. J., Jackson, J. C., Kratz, L., Marcuse, E. K., McPhillips, H. A., Shugerman, R. P., et al. (2003). Teaching professionalism to residents. Academic Medicine, 78(1), 26-34. Misch, D. A. (2002). Evaluating physicians' professionalism and humanism: the case for humanism "connoisseurs". Academic Medicine, 77(6), 489-495. Papadakis, M. A., Hodgson, C. S., Teherani, A., & Kohatsu, N. D. (2004). Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. Academic Medicine, 79(3), 244-249. Topps, D., Evans, R., Khan, L., & Hays, R. B. (2004). Personal Digital Assistants in Vocational Training (Report, Adobe PDF). Townsville, Queensland: James Cook University. Van De Camp, K., Vernooij-Dassen, M. J., Grol, R. P., & Bottema, B. J. (2004). How to conceptualize professionalism: a qualitative study. Medical Teacher, 26(8), 696-702. Veloski, J. J., Fields, S. K., Boex, J. R., & Blank, L. L. (2005). Measuring professionalism: a review of studies with instruments reported in the literature between 1982 and 2002. Academic Medicine, 80(4), 366-370. Wass, V. (2005). The changing face of assessment: swings and roundabouts. British Journal General Practice, 55(515), 420-422.
URI:	http://wrap.warwick.ac.uk/id/eprint/4418

Request changes to a record

Actions (login required)

View Item