The Library
Comparing holistic and analytic scoring methods : issues of validity and reliability.
Tools
Harsch, Claudia and Martin, Guido (2013) Comparing holistic and analytic scoring methods : issues of validity and reliability. Assessment in Education: Principles, Policy & Practice, Volume 20 (Number 3). pp. 281-307. doi:10.1080/0969594X.2012.742422 ISSN 0969-594X.
Research output not available from this repository.
Request-a-Copy directly from author or use local Library Get it For Me service.
Official URL: http://dx.doi.org/10.1080/0969594X.2012.742422
Abstract
This paper explores issues of rating quality when assessing writing in a level-specific approach, i.e. tasks targeting specific proficiency levels. We investigate whether holistic scores could mask deviances in how underlying descriptors are interpreted and applied by the raters, as such deviances could compromise rating validity. We conducted a study with six raters, comparing a holistic approach with a combined approach whereby an analytic score for each descriptor was collected complementary to the holistic judgements. The results confirmed the initial hypothesis that holistic scores may mask deviances in how descriptors are applied. To monitor rating quality and enhance rating validity, we therefore recommend a complementary approach combining holistic scores with analytic, descriptor-focused scores. We illustrate the applicability of this approach by a successful implementation during rater training. Our findings contribute towards enhancing rater consistency and improving rating validity.
Item Type: | Journal Article | ||||
---|---|---|---|---|---|
Subjects: | H Social Sciences > H Social Sciences (General) | ||||
Divisions: | Faculty of Social Sciences > Centre for Applied Linguistics | ||||
Journal or Publication Title: | Assessment in Education: Principles, Policy & Practice | ||||
Publisher: | Routledge | ||||
ISSN: | 0969-594X | ||||
Official Date: | 2013 | ||||
Dates: |
|
||||
Volume: | Volume 20 | ||||
Number: | Number 3 | ||||
Page Range: | pp. 281-307 | ||||
DOI: | 10.1080/0969594X.2012.742422 | ||||
Status: | Peer Reviewed | ||||
Publication Status: | Published | ||||
Access rights to Published version: | Restricted or Subscription Access |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |