The Library
Is the assumption of equal distances between global assessment categories used in borderline regression valid?
Tools
Tools
Tools
McGown, Patrick J., Brown, Celia A., Sebastian, Ann, Le, Ricardo, Amin, Anjali, Greenland, Andrew and Sam, Amir H. (2022) Is the assumption of equal distances between global assessment categories used in borderline regression valid? BMC Medical Education, 22 (1). 708. doi:10.1186/s12909-022-03753-5 ISSN 1472-6920.
|
PDF
WRAP-Is-the-assumption-of-equal-distances-between-global-assessment-categories-used-in-borderline-regression-valid-2022.pdf - Published Version - Requires a PDF viewer. Available under License Creative Commons Attribution 4.0. Download (1575Kb) | Preview |
Official URL: http://dx.doi.org/10.1186/s12909-022-03753-5
Abstract
Background
Standard setting for clinical examinations typically uses the borderline regression method to set the pass mark. An assumption made in using this method is that there are equal intervals between global ratings (GR) (e.g. Fail, Borderline Pass, Clear Pass, Good and Excellent). However, this assumption has never been tested in the medical literature to the best of our knowledge. We examine if the assumption of equal intervals between GR is met, and the potential implications for student outcomes.
Methods
Clinical finals examiners were recruited across two institutions to place the typical ‘Borderline Pass’, ‘Clear Pass’ and ‘Good’ candidate on a continuous slider scale between a typical ‘Fail’ candidate at point 0 and a typical ‘Excellent’ candidate at point 1. Results were analysed using one-sample t-testing of each interval to an equal interval size of 0.25.
Secondary data analysis was performed on summative assessment scores for 94 clinical stations and 1191 medical student examination outcomes in the final 2 years of study at a single centre.
Results
On a scale from 0.00 (Fail) to 1.00 (Excellent), mean examiner GRs for ‘Borderline Pass’, ‘Clear Pass’ and ‘Good’ were 0.33, 0.55 and 0.77 respectively.
All of the four intervals between GRs (Fail-Borderline Pass, Borderline Pass-Clear Pass, Clear Pass-Good, Good-Excellent) were statistically significantly different to the expected value of 0.25 (all p-values < 0.0125).
An ordinal linear regression using mean examiner GRs was performed for each of the 94 stations, to determine pass marks out of 24. This increased pass marks for all 94 stations compared with the original GR locations (mean increase 0.21), and caused one additional fail by overall exam pass mark (out of 1191 students) and 92 additional station fails (out of 11,346 stations).
Conclusions
Although the current assumption of equal intervals between GRs across the performance spectrum is not met, and an adjusted regression equation causes an increase in station pass marks, the effect on overall exam pass/fail outcomes is modest.
| Item Type: | Journal Article | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Subjects: | L Education > LB Theory and practice of education R Medicine > R Medicine (General) R Medicine > RA Public aspects of medicine |
||||||||
| Divisions: | Faculty of Science, Engineering and Medicine > Medicine > Warwick Medical School | ||||||||
| Library of Congress Subject Headings (LCSH): | Medical education , Medical education policy , Medical students, Medical students -- Examinations -- Standards, Academic achievement -- Evaluation | ||||||||
| Journal or Publication Title: | BMC Medical Education | ||||||||
| Publisher: | BioMed Central Ltd. | ||||||||
| ISSN: | 1472-6920 | ||||||||
| Official Date: | 5 October 2022 | ||||||||
| Dates: |
|
||||||||
| Volume: | 22 | ||||||||
| Number: | 1 | ||||||||
| Number of Pages: | 10 | ||||||||
| Article Number: | 708 | ||||||||
| DOI: | 10.1186/s12909-022-03753-5 | ||||||||
| Status: | Peer Reviewed | ||||||||
| Publication Status: | Published | ||||||||
| Access rights to Published version: | Open Access (Creative Commons) | ||||||||
| Date of first compliant deposit: | 25 October 2022 | ||||||||
| Date of first compliant Open Access: | 25 October 2022 |
Request changes or add full text files to a record
Repository staff actions (login required)
 |
View Item |
Downloads
Downloads per month over past year
