Brooks, Valerie. (2012) Marking as judgment. Research papers in Education , Vol.27 (No.1). pp. 63-80. ISSN 0267-1522

PDF WRAP_Brooks_9270464-ie-090210-marking_as_judgmentcomplete_final_version.pdf - Accepted Version - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader Download (294Kb)

Abstract

An aspect of assessment which has received little attention compared with perennial concerns, such as standards or reliability, is the role of judgment in marking. This paper explores marking as an act of judgment, paying particular attention to the nature of judgment and the processes involved. It brings together studies which have explored marking from a psychological perspective for the purpose of critical discussion of the light they shed on each other and on the practice of marking. Later stages speculate on recent developments in psychology and neuroscience which may cast further light on educational assessment.

Item Type:	Journal Article
Subjects:	L Education > LB Theory and practice of education B Philosophy. Psychology. Religion > BF Psychology
Divisions:	Faculty of Social Sciences > Institute of Education
Library of Congress Subject Headings (LCSH):	Grading and marking (Students) -- Great Britain, Educational tests and measurements -- Great Britain, Examinations -- Interpretation, Achievement motivation, Educational psychology
Journal or Publication Title:	Research papers in Education
Publisher:	Routledge
ISSN:	0267-1522
Date:	11 November 2012
Volume:	Vol.27
Number:	No.1
Page Range:	pp. 63-80
Identification Number:	10.1080/02671520903331008
Status:	Peer Reviewed
Publication Status:	Published
Access rights to Published version:	Restricted or Subscription Access
References:	Baddeley, A. 1998. Working memory. Life Sciences 321: 167-173. Bridges, D., J. Elliott, and A. McKee. 1995. Competence-based Higher Education and the Standards Methodology. Norwich, University of East Anglia: the Employment Department and the Engineering Council. Chapman, G. B. 2005. The psychology of medical decision making. In Blackwell Handbook of judgment and decision making, eds. D. Koehler and N. Harvey, 585-603. Oxford: Blackwell. Cooksey, R., P. Freebody, and C. Wyatt-Smith. 2007. Assessment as judgment-in-context: Analysing how teachers evaluate students‟ writing. Educational Research and Evaluation 13: 401-434. Crisp. V. 2008. Exploring the nature of examiner thinking during the process of examination marking. Cambridge Journal of Education 38: 247-264. Dennis, I., S. Newstead, and D. Wright. 1996. A new approach to exploring biases in educational assessment. British Journal of Psychology 87: 515-534. Ecclestone, K. 2001. „I know a 2:1 when I see it‟: Understanding criteria for degree classifications in franchised university programmes. Journal of Further and Higher Education 25: 301-313. Elander, J. 2004. Student assessment from a psychological perspective. Psychology Learning and Teaching 3: 114-121. Elander, J., and D. Hardman. 2002. An application of judgment analysis to examination marking in psychology. British Journal of Psychology 93: 303-328. Eraut, M. 2000. Non-formal learning and tacit knowledge in professional work. British Journal of Educational Psychology 70: 113-136. Ericsson, K. A. 2002. Towards a procedure for eliciting verbal expression of non-verbal experience without reactivity: Interpreting the verbal overshadowing effect within the theoretical framework for protocol analysis. Applied Cognitive Psychology 16: 981-987. Evans, J. 1993. Bias and rationality. In Rationality: Psychological and philosophical perspectives, eds. K. Manktelow, and D. Over, 6-30. London: Routledge. Fessey, C. 2002. Capturing expertise in the development of practice: Methodology and approaches. Learning in Health and Social Care 1: 47-58. Garry, J., M. McCool, and S. O‟Neill. 2005. Are moderators moderate? Testing the „anchoring and adjustment‟ hypothesis in the context of marking Politics exams. Politics 25: 191-200. Gilovich, T., and D. Griffin. 2002. Introduction – heuristics and biases: Then and now. In Heuristics and biases: The psychology of intuitive judgment, ed. T. Gilovich, D. Griffin, and D. Kahneman, 1-17. Cambridge: Cambridge University Press. Gipps, C., and P. Murphy. 1994. A fair test? Assessment, achievement and equity. Buckingham: Open University Press. Grimley, M., H. Dahraei, and R. Riding. 2008. The relationship between anxiety-stability, working memory and cognitive style. Educational Studies 34: 213- 223. Hall, K., and A. Harding. 2002. Level descriptions and teacher assessment in England: Towards a community of assessment practice. Educational Research 44: 1-15. Hardman, D. 2009. Judgment and decision making: Psychological perspectives. Chichester: BPS Blackwell. Harlen, W. 2004. A systematic review of the evidence of reliability and validity of assessment by teachers used for summative purposes. In Research Evidence in Education Library. EPPI-Centre, London: Institute of Education. Harlen, W. 2005. Trusting teachers‟ judgement: Research evidence of the reliability and validity of teachers‟ assessment used for summative purposes. Research Papers in Education 20: 245-270. Hartley, J., M. Trueman, L. Betts, and L. Brodie. 2006. What price presentation? The effects of typographic variables on essay grades. Assessment and Evaluation in Higher Education 31: 523-534. Hay, P., and D. Macdonald. 2008. (Mis)appropriations of criteria and standards-referenced assessment in a performance-based subject. Assessment in Education: Principles, Policy and Practice 15: 153-168. Immordino-Yang, M., and A. Damasio. 2007. We feel, therefore we learn: The relevance of affective and social neuroscience to Education. Mind, Brain and Education 1: 3-10. Jackson, P. 2002. The constructs in people‟s heads. In The role of constructs in psychological and educational measurement, ed. H. Braun, D. Jackson, and D. Wiley. Mahwah, NJ: Lawrence Erlbaum. Jonsson, A., and G. Svingby. 2007. The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review 2: 130-144. Keren, G. and K. Teigen. 2004. Yet another look at the heuristics and biases approach. In Blackwell handbook of judgment and decision making, ed. D. Koehler, and N. Harvey, 89-109. Oxford: Blackwell. Laming, D. 2004. Human judgement: The eye of the beholder. London: Thomson. Leighton, J. P. 2004. Avoiding misconception, misuse and missed opportunities: The collection of verbal reports in educational achievement testing. Educational Measurement: Issues and Practice 23: 6-15. Marshall, B. 2000. English teachers - the unofficial guide: Researching the philosophies of English teachers. London: RoutledgeFalmer. Meier, S., B. Rich, and J. Cady. 2006. Teachers‟ use of rubrics to score non-traditional tasks: Factors related to discrepancies in scoring. Assessment in Education: Principles, Policy and Practice 13: 69-95. Milanovic, M., N. Saville, and S. Shuhong. 1996. A study of the decision-making behaviour of composition markers. In Studies in Language Testing 3: Performance testing, cognition and assessment, ed. M. Milanovic, and N. Saville, 92-114. Cambridge: Cambridge University Press. Morgan, C. 1996. The teacher as examiner: The case of Mathematics coursework. Assessment in Education: Principles, Policy and Practice 3: 353-374. Murphy, R.J.L. 1978. Reliability of marking in eight GCE examinations. British Journal of Educational Psychology 48: 196-200. Murphy, R.J.L. 1979. Removing the marks from examination scripts before re-marking them: Does it make any difference? British Journal of Educational Psychology 49: 73-78. Newton, P. 1996. The reliability of marking of General Certificate of Secondary Education scripts: Mathematics and English. British Educational Research Journal 22: 405-420. Rottenstreich, Y. and S. Shu. 2004. The connections between affect and decision making: Nine resulting phenomena. In Blackwell handbook of judgment and decision making, ed. D. Koehler, and N. Harvey, 444-463. Oxford: Blackwell. Pollitt, A., and N. Murray. 1996. What raters really pay attention to. In Studies in Language Testing 3: Performance testing, cognition and assessment, ed. M. Milanovic, and N. Saville, 74-91. Cambridge: Cambridge University Press. Price, M., and C. Rust. 1999. The experience of introducing a common criteria assessment grid across an academic department. Quality in Higher Education 5: 133-144. Quality Assurance Agency for Higher Education (QAA). 2006. Code of practice for the assurance of academic quality and standards in higher education. Gloucester: Quality Assurance Agency for Higher Education. Quality Assurance Agency for Higher Education (QAA). 2009. Thematic enquiries into concerns about academic quality and standards in higher education in England: Final report. Gloucester: Quality Assurance Agency for Higher Education. Sadler, D.R. 2005. Interpretations of criteria-based assessment and grading in higher education. Assessment and Evaluation in Higher Education 30: 175-194. Scharaschkin, A., and J. Baird. 2000. The effects of consistency of performance on A Level examiners‟ judgements of standards. British Educational Research Journal 26: 343-358. Sherman, S., R. Cialdini, D. Schwartzman, and K. Reynolds. 2002. Imagining can heighten or lower the perceived likelihood of contracting a disease: The mediating effect of ease of imagery. In Heuristics and biases: The psychology of intuitive judgment, ed. T. Gilovich, D. Griffin, and D. Kahneman, 98-102. Cambridge: Cambridge University Press. Slovic, P., M. Finucane, E. Peters, and D. MacGregor. 2002. The affect heuristic. In Heuristics and biases: The psychology of intuitive judgment, ed. T. Gilovich, D. Griffin, and D. Kahneman, 397-420. Cambridge: Cambridge University Press. Suto, I., and J. Greatorex. 2008. What goes through an examiner‟s mind? Using verbal protocols to gain insight into the GCSE marking process. British Educational Research Journal 34: 213-233. Szpara, M., and C. Wylie. 2005. National Board for Professional Teaching Standards assessor training: Impact of bias reduction exercises. Teachers College Record 107: 803-841. Tattersall, K. 2008. Speech at the Ofqual launch event. National Motorcycle Museum, Solihull, UK, 16 May. Tversky, A., and D. Kahneman. 2002. Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. In Heuristics and biases: The psychology of intuitive judgment, ed. T. Gilovich, D. Griffin, and D. Kahneman, 19-48. Cambridge: Cambridge University Press. Van Someren, M., Y. Barnard, and J. Sandberg. 1994. The think aloud method: A practical guide to modelling cognitive processes. London: Academic Press. Vaughan, C. 1992. Holisitic assessment: What goes on in the rater‟s mind? In Assessing second language writing in academic contexts, ed. L. Hamp-Lyons, 111-125. Norwood, NJ: Ablex. Wenger, E. 1998. Communities of practice: Learning, meaning and identity. Cambridge: Cambridge University Press. Wiliam, D. 1992. Some technical issues in assessment: A user‟s guide. British Journal of Curriculum and Assessment 2: 11-20. Wiliam, D. 1996. Standards in examinations: A matter of trust? The Curriculum Journal 7: 293-306. Wilmut, J., R. Wood, and R. Murphy. 1996. A review of research into the reliability of examinations: Discussion paper prepared for the School Curriculum and Assessment Authority. Nottingham: University of Nottingham. http://www.nottingham.ac.uk/education/centres/cdell/pdf-reportsrelexam/relexam.pdf (accessed February 5, 2003). Wolf, A. 1995. Competence-based assessment. Buckingham: Open University Press. Wyatt-Smith, C., and G. Castleton. 2005. Examining how teachers judge student writing: An Australian case study. Journal of Curriculum Studies 37: 131-154.
URI:	http://wrap.warwick.ac.uk/id/eprint/2812

Request changes to a record

Actions (login required)

View Item