The Library
Comparing single-best-answer and very-short-answer questions for the assessment of applied medical knowledge in 20 UK medical schools : cross-sectional study
Tools
Sam, Amir H., Westacott, Rachel, Gurnell, Mark, Wilson, Rebecca, Meeran, Karim and Brown, Celia A. (2019) Comparing single-best-answer and very-short-answer questions for the assessment of applied medical knowledge in 20 UK medical schools : cross-sectional study. BMJ Open, 9 (9). e032550. doi:10.1136/bmjopen-2019-032550 ISSN 2044-6055.
|
PDF
WRAP-comparing-single-best-answer-questions-applied-knowledge-Brown-2019.pdf - Published Version - Requires a PDF viewer. Available under License Creative Commons: Attribution-Noncommercial 4.0. Download (362Kb) | Preview |
Official URL: https://doi.org/10.1136/bmjopen-2019-032550
Abstract
Objectives The study aimed to compare candidate performance between traditional best-of-five single-best-answer (SBA) questions and very-short-answer (VSA) questions, in which candidates must generate their own answers of between one and five words. The primary objective was to determine if the mean positive cue rate for SBAs exceeded the null hypothesis guessing rate of 20%. Design This was a cross-sectional study undertaken in 2018. Setting 20 medical schools in the UK. Participants 1417 volunteer medical students preparing for their final undergraduate medicine examinations (total eligible population across all UK medical schools approximately 7500). Interventions Students completed a 50-question VSA test, followed immediately by the same test in SBA format, using a novel digital exam delivery platform which also facilitated rapid marking of VSAs. Main outcome measures The main outcome measure was the mean positive cue rate across SBAs: the percentage of students getting the SBA format of the question correct after getting the VSA format incorrect. Internal consistency, item discrimination and the pass rate using Cohen standard setting for VSAs and SBAs were also evaluated, and a cost analysis in terms of marking the VSA was performed. Results The study was completed by 1417 students. Mean student scores were 21 percentage points higher for SBAs. The mean positive cue rate was 42.7% (95% CI 36.8% to 48.6%), one-sample t-test against ≤20%: t=7.53, p<0.001. Internal consistency was higher for VSAs than SBAs and the median item discrimination equivalent. The estimated marking cost was £2655 ($3500), with 24.5 hours of clinician time required (1.25 s per student per question). Conclusions SBA questions can give a false impression of students’ competence. VSAs appear to have greater authenticity and can provide useful information regarding students’ cognitive errors, helping to improve learning as well as assessment. Electronic delivery and marking of VSAs is feasible and cost-effective.
Item Type: | Journal Article | ||||||
---|---|---|---|---|---|---|---|
Subjects: | R Medicine > R Medicine (General) R Medicine > RC Internal medicine |
||||||
Divisions: | Faculty of Science, Engineering and Medicine > Medicine > Warwick Medical School > Health Sciences > Population, Evidence & Technologies (PET) Faculty of Science, Engineering and Medicine > Medicine > Warwick Medical School |
||||||
SWORD Depositor: | Library Publications Router | ||||||
Library of Congress Subject Headings (LCSH): | Medicine -- Examinations, questions, etc., Medical sciences -- Examinations, questions, etc., Medical students -- Rating of, Clinical medicine -- Examinations, questions, etc., Medical colleges -- Graduate work -- Examinations , Medicine -- Ability testing | ||||||
Journal or Publication Title: | BMJ Open | ||||||
Publisher: | BMJ Publishing Group | ||||||
ISSN: | 2044-6055 | ||||||
Official Date: | 26 September 2019 | ||||||
Dates: |
|
||||||
Volume: | 9 | ||||||
Number: | 9 | ||||||
Article Number: | e032550 | ||||||
DOI: | 10.1136/bmjopen-2019-032550 | ||||||
Status: | Peer Reviewed | ||||||
Publication Status: | Published | ||||||
Access rights to Published version: | Open Access (Creative Commons) | ||||||
Date of first compliant deposit: | 27 September 2019 | ||||||
Date of first compliant Open Access: | 27 September 2019 | ||||||
RIOXX Funder/Project Grant: |
|
||||||
Related URLs: |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year