SESSION 3E

Mohammed Sameer & Janette Greenwood, CET University of Sydney
Maintaining rater reliability: Rater training and standardisation

Abstract
Assessment in education plays a pivotal role. It influences both how students learn and how teachers teach. In light of this role, the rating of writing and speaking assessment tasks, where rater subjectivity can be an issue, must encompass stringent procedures to ensure awarded marks are reflective of students’ performance. Following these procedures becomes more important in courses that have a significant number of raters (or assessors) where discrepancies in rater judgements may exist.

The Centre for English Teaching has developed a number of procedures to address the issue of rater subjectivity. This system for standardising productive skills has been developed as a result of the need for consistent final results for university admission and because a large number of teachers are employed specifically for the last 10 weeks of Direct Entry Courses when the assessment tasks are conducted.

This session aims to provide insight into the procedures implemented to make assessment as reliable and valid as possible minimising rater subjectivity.

Bio
Jannette Greenwood has been a teacher at the University of Sydney Centre for English Teaching for nearly 20 years and previously taught at the British Council Singapore for 6 years. For the past 12 years she has been involved in the Assessment of the Direct Entry program, writing and developing the University Direct Entry suite of tests. She is part of the standardisation team known as the Assessment Working Group. Jannette has an MA (App.Ling), which included testing units, and the Dip.TEFLA. She was an IELTS Writing and Speaking examiner for 20 years and wrote the reading component of “Focusing on IELTS – Reading and Writing Skills 2nd ed.” published by Macmillan.

Mohammed Sameer has been designing assessment tasks since 2005 when he was an Assistant Lecturer (Linguistics) at Fiji’s University of the South Pacific. He has an MA in Linguistics and a PhD in Education. His thesis focussed on determining needs of academic English students. At the University of Sydney, Centre for English Teaching, he is part of the Assessment Working Group, primarily responsible for writing assessment tasks and maintaining rater standards through processes of standardisation and moderation. He has been an IELTS writing examiner and is currently a speaking examiner at University of Technology, Sydney.