The Library
Adapting CEF-descriptors for rating purposes: Validation by a combined rater training and scale revision approach
Tools
Tools
Tools
Harsch, Claudia and Martin, Guido (2012) Adapting CEF-descriptors for rating purposes: Validation by a combined rater training and scale revision approach. Assessing Writing, Vol. 17 (No. 4). pp. 228-250. doi:10.1016/j.asw.2012.06.003
Research output not available from this repository, contact author.
Official URL: http://www.sciencedirect.com/science/article/pii/S...
Abstract
We explore how a local rating scale can be based on the Common European Framework CEF-proficiency scales. As part of the scale validation (Alderson, 1991; Lumley, 2002), we examine which adaptations are needed to turn CEF-proficiency descriptors into a rating scale for a local context, and to establish a practicable method to revise the initial scale-draft while simultaneously training raters for trialling the rating scale. We propose a data-driven approach focusing on the descriptors, involving raters in the scale revision as part of the scale validation process (Barkaoui, 2010a; Cohen, 1994). Rating data were analysed during a two-month period, employing 19 writing tasks and 13 raters; individual descriptors and raters were monitored for consistency. Our approach facilitated identifying causes of low agreement, let it be raters or scale
wording, which in turn allowed targeted feedback to raters and revisions of scale-wording where indicated. We conclude with a summary of necessary adaptations of CEF-descriptors for rating purposes, and we outline further steps to validate the revised rating scale. We aim to contribute to improve rater training and rating scale development based on the CEF, since the literature for ‘real’ test development contexts seems rather scarce (Barkaoui, 2010b; Knoch, 2011a,b).
| Item Type: | Journal Article | ||||
|---|---|---|---|---|---|
| Subjects: | L Education > L Education (General) P Language and Literature > PE English |
||||
| Divisions: | Faculty of Social Sciences > Centre for Applied Linguistics | ||||
| Journal or Publication Title: | Assessing Writing | ||||
| Publisher: | Elsevier | ||||
| ISSN: | 1075-2935 | ||||
| Official Date: | 2012 | ||||
| Dates: |
|
||||
| Volume: | Vol. 17 | ||||
| Number: | No. 4 | ||||
| Number of Pages: | 23 | ||||
| Page Range: | pp. 228-250 | ||||
| DOI: | 10.1016/j.asw.2012.06.003 | ||||
| Status: | Peer Reviewed | ||||
| Publication Status: | Published | ||||
| Access rights to Published version: | Restricted or Subscription Access |
Request changes or add full text files to a record
Repository staff actions (login required)
 |
View Item |
