SESSION 4B

John Gardiner & Tony Hickey, CET University of Sydney
Rubric Reiterations

Abstract
By constructively aligning course content with assessment and marking rubrics, direct-entry course developers are more likely to achieve their goals. This session presents the considerations and challenges when developing a test rubric and some ways to address these challenges. Through several editing, feedback and benchmarking processes at the Centre for English Teaching (CET), the marking rubric for the Direct Entry Course (DEC) assessment tasks have been created and modified. These processes include both internal and external validity to maintain high marker reliability and to reconcile any discrepancies. The criteria and design elements of the rubric were designed to be consistent from one assessment task to another. However, as issues such as curriculum design, assessment changes and university entry requirements changed, adjusting the rubric became increasingly problematic. Attempts to address these dilemmas and the importance of considering feedback from several stakeholder perspectives will be discussed in this presentation. One solution, for instance, was to give more agency to students by linking writing achievement with calibration accuracy (MacArthur & Philippakos 2013; Valdez, 2013). Also, feedback from the analysis of external bench markers added to the complex considerations in the rubric development. The final product is the result of all these reiterations and considerations. The implications of rubric format, wording, scaling and design choices are rarely discussed in the literature, but these elements could have significant impacts on the markers and students. Furthermore, the validity implications of course content, assessment and rubric alignment are relevant for all stakeholders in direct entry university courses.

Bio
John Gardiner is a teacher at the Centre for English Teaching (CET), University of Sydney, Australia. He has extensive teaching and curriculum development experience on direct entry post-graduate EAP programs. As a member of the Assessment Working Group at CET, he has developed tests and rubrics for high-stakes direct-entry programs. His published research on the test-taking experience of students as well as other relevant publications have provided a foundation for the development of his course material. John has presented at overseas and Australian ESL and testing conferences. His research interests include language testing, action research, curriculum design, and course development.

Tony Hickey has worked in EFL in Sydney for over 20 years. He manages the direct entry courses at the University of Sydney Centre for English Teaching with a focus on assessment practices. He coordinates the Centre’s assessment working group on assessment design, standardisation and moderation