Assessment Symposium 2024

This year, our keynote speaker is a dedicated educator, who is committed to providing meaningful and engaging learning experiences. 

This year, our keynote speaker is a dedicated educator, who is committed to providing meaningful and engaging learning experiences. 

Bio
Pranit has been teaching in higher education institutions in Australia and overseas since 2000. He is highly regarded by his students and colleagues and is frequently nominated for various awards. He is passionate about student engagement and success, and attributes this to the considerable time spent teaching within the enabling and pathways sector, including a significant time on the executive of the National Association of Enabling Educators of Australia (NAEEA). He even spoke at a UECA PD Fest in Sydney almost 10 years ago!

These days Pranit teaches various networking and cybersecurity courses in both undergraduate and postgraduate courses. He recognises the important role assessments play towards student engagement and success and attempts to design assessments that are meaningful, transferrable, flexible and inclusive.

Find out more:

LinkedIn: linkedin.com/in/dr-pranit-anand
Twitter/X: @pranit_a

Register Here

With thanks to our Event Sponsors

---

Registration
Foyer
9:30 – 10:00 AEST

Refreshments & coffee/tea served.

Welcome & Acknowledgement of Country
The Auditorium, Level 2
10:00 – 10:10 AEST

Macgregor Haines, Manager Learning and Teaching Excellence, Monash College
Simon Winetroube, UECA President and Director English, Curtin University

Keynote – Finding trust: embedding meaningful future-ready knowledge, skills and attitudes through personalised learning in an AI powered world
The Auditorium, Level 2
10:10 – 10:50 AEST

Dr Pranit Anand
PhD, MACS, SFHEA, FHERDSA
Senior Lecturer/Education Innovation Coordinator, School of Information Systems and Technology Management, UNSW Business School

Abstract
The challenges to academic integrity from emerging technologies is obvious, while many higher education courses have attempted to counter these challenges in various ways, there is a risk that we may be inadvertently disconnecting with our students and even more concerning is the risk to assurance of learning through these initiatives.

Artificial intelligence provides significant opportunities for businesses and industry, and as we grapple with harnessing the power of AI for our society, we will need well-informed, and competent graduates to lead this transformation in responsible and ethical directions. Preventing students from using these tools during their formative years is unlikely to help them develop these skills. During this talk Pranit will share ideas that he has used in his own courses to engage students in their learning process, carefully curating the learning journey with A.I. Assessments in many of his courses reflect the expectations of the real world of work and involve conversational and relational elements.

Students engage in the use of A.I. within in-class discussions, and various formative activities. Assessments are designed to elicit students’ ability to collaboratively engage with A.I. to develop meaningful solutions to authentic challenges and identify opportunities and challenges with existing configurations. The higher stakes assessments involve conversations about what they have learned and how they will be able to apply these in the real world. Some of the initial evaluation data indicates students enjoy these activities and assessments, and relish the opportunity to engage in deep, meaningful conversations with their peers and teachers.

Keynote – Interactive and Collaborative Q&A Session
The Auditorium, Level 2
10:50 – 11:25 AEST

Chair: Macgregor Haines, Manager Learning and Teaching Excellence, Monash College

Tracing Success: A Sector Snapshot of Direct Entry Programs
The Auditorium, Level 2
11:30 – 12:00 AEST

Katherine Olston, Director, Centre for English Teaching & Head, Learning Hub, University of Sydney
Dr.Pamela Humphreys, Associate Professor, Macquarie University College
Simon Winetroube, Associate Professor, Director, Curtin English
Heather Thomas, Deputy General Manager/Director Global Programs, UOW College
Prof. Thomas Roche, Pro Vice-Chancellor (Academic Quality) at Southern Cross University
Dr Joshua Swift, Project Coordinator within the CET, University of Sydney
Chair: Macgregor Haines, Manager Learning and Teaching Excellence, Monash College

Abstract
English language proficiency is understood to be an important factor in the academic success for English as Additional Language (EAL) students studying in universities in Australia (Humphreys et al., 2012; Roche, 2017) and English Medium Instruction (EMI) universities abroad (Rose et al., 2020; Soruç et al., 2021). Direct Entry (DE) programs allow students to develop their English language skills and increase their preparedness to study in Australian universities (English Australia, 2017). Upon successful completion of the DE program, students meet university admission requirements and are granted ‘direct entry’ to their chosen degree course.

To ensure that DE curricula offer participants the best chance of success, providers employ several mechanisms to evidence program efficacy. This is also a regulatory requirement, with the ELICOS Standards (2018) requiring that DE providers have measures to compare admission criteria for degree programs with the outcomes of their offering. Tracer studies (TS), which track previous DE students’ performance in their subsequent studies, are one way this can be achieved. Whilst the use of TS in understanding the connection between higher education and graduate outcomes has been long-established, there is a lack of published research surrounding their use in evaluating the link between DE programs and students’ performance in their degree studies.

This project evaluates the use of tracer studies at University English Centres Australia (UECA) member institutions and the utility of their outcomes in informing improvements in DE programs. In this presentation, we will provide an overview of this tracer study project, and a work-in-progress update.

Bios
Katherine Olston is the Director at the Centre for English Teaching (CET) and Head of the Learning Hub at the University of Sydney. She oversees pre-entry English language and academic skills pathways courses and post-entry academic support programs in mathematics and academic skills and language development. Her research interests include academic learning support as well as conceptualisations and enactment of teaching, learning and assessment of the graduate qualities in in higher education.

Joshua Swift has over 10 years of experience in medical science and higher education, and 4 years’ experience in strategy consulting for both national and global organisations. Highlights include a cancer research start-up and strategic projects within the University of NSW, the University of Sydney, and ASX-listed companies. His career continues to focus on making a tangible difference to society in the areas of oncology and higher education.

Lunch
Level 5
12:00 – 12:50 AEST

Lunch is provided.

Session 1A – Enhancing Language Assessment Security through Teacher Training and Collaborative Design
Room 5.61, Level 5
12:50 – 13:35 AEST

Phuong Tran, Manager of Assessment, Monash College
Mark Rooney, Program Leader, Monash College
Chair: Jenni Coster

Abstract
Maintaining the security of language assessments poses significant challenges, particularly with the ease of online content sharing today. This presentation explores a comprehensive approach to enhancing the security and validity of English Reading and Listening assessments through targeted teacher training and collaborative test design. This session will be divided into three segments. First, we will discuss a two-day teacher training workshop series designed to equip teachers with essential skills in assessment development. This includes crafting texts for reading and listening assessments, leveraging technologies for text writing, writing comprehension questions, and interpreting Moodle statistics to refine test items. Interactive hands-on activities and continuous feedback were integral to the training, ensuring practical skill application.

Next, we highlight how the trained teachers applied their newfound skills to produce multiple versions of reading and listening assessments. This step is crucial for regularly updating assessments on the learning management system, thereby mitigating the risks associated with test content exposure and sharing by students. A teacher will contribute practical examples of how they developed the assessments. Finally, all three presenters will share the lessons learned from this collaborative endeavour. We will discuss the impact of continuous professional development on assessment design, the challenges faced in maintaining test integrity, and the strategies implemented to overcome these challenges.

Our presentation aims to provide valuable insights into this collaborative process. By empowering teachers, we strive to ensure the validity and security of online language assessments.

Bios
Phuong Tran is Manager of Assessment from Learning Design and Digital Innovation. She specialises in educational assessment and measurement. Phuong has extensive experience in assessment and rubrics development, validation and benchmarking, language teaching, curriculum development and management, and teacher training. Her current interests are transition education and quality assessment design in the face of generative artificial intelligence.

Mark Rooney is currently a Program Leader at Monash College English, where he has also worked as a Learning Skills Adviser and Teacher Developer. His current areas of interest are building teacher autonomy and the role of mentoring in education.

Session 1B – Reassessing Rubrics in the age of genAI
Room 5.64 Level 5
12:50 – 13:35 AEST

Stuart Parker, Academic Communication Skills & Academic English Skills Coordinator, ACU University
Chair: MC Rep

Abstract
Academic writing is an integral part of university study, both in the expression of thoughts and ideas and in the synthesis of knowledge. Indeed, it could be argued that the careful and deliberate composition of language into ‘artifacts’ is the purest form of understanding available. The emergence of generative AI writing tools, however, has necessitated a reappraisal of how assessment tasks are structured and delivered and, in turn, how rubrics can best reflect these adaptions.

Trust but Verify: Rubrics in the age of genAI is an action research project being conducted within the ACU Foundation Studies’ Academic Communication and Academic English units with 3 main areas of focus:

1. How rubrics can be revised to accommodate the emergence of genAI (to address both acceptable and unacceptable uses of genAI where applicable)
2. How to encourage students through rubric descriptors to value their own voice and ideas against the lure of genAI’s instant output and seemingly effortless solutions
3. What accompanying teaching materials and approaches can best highlight the inherent limitations of genAI writing tools in developing academic stances and constructing language

The proposed session will offer an overview of the action research project, including its rationale, construct, and the initial teacher and student feedback garnered through surveys and focus group discussions. Rubric samples and accompanying teaching materials will also be presented.

Attendees will be invited to consider what common/traditional rubric descriptors are readily achievable by genAI and how these might best be modified or indeed replaced.

Bio
Stuart has been at the Australian Catholic University for 15 years and in that time has worked as an Academic Manager, Coordinator, Lecturer in Charge and teacher in such programs as Foundation Studies, English for Academic Purposes, Communication Skills and IELTS/PTE Preparation courses. Currently, he is the coordinator for both domestic and international cohorts in two units within ACU’s Education Pathways: Academic Communication Skills and Academic English Skills. In 2023, he participated in UECA’s Melbourne Symposium as a member of the expert panel and as a presenter for the session Learning Strategies Interrupted.

Session 1C – Language Testing and Student Employability
Room 5.30, Level 5
12:50 – 13:35 AEST

Dr Patrick Pheasant, Head of Education Partnerships Australia & New Zealand, LanguageCert
Chair: MC Rep

Abstract
This presentation explores the critical role of English language assessment in facilitating the successful integration of international students into the global workforce. It emphasises the importance of standardised language tests as a reliable and objective measure of language proficiency, crucial for both academic pursuits and professional success.

The presentation will delve into the following key areas:

1. Standardised English language tests as a gateway to global opportunities: Examination of how these tests provide a common benchmark for evaluating language skills, enabling employers to make informed hiring decisions and ensuring a level playing field for candidates from diverse linguistic backgrounds.
2. Correlation between language proficiency and employability: Investigation of research demonstrating the positive impact of strong English language skills on career prospects for international graduates, highlighting the importance of language assessment as a predictor of workplace success.
3. The role of language tests in professional development: Exploration of how language assessments can serve as a valuable tool for identifying areas for improvement and guiding targeted language training, empowering international students to enhance their language skills and maximise their employability.
4. Supporting inclusivity through fair and transparent assessment: Discussion of the rigorous development and validation processes involved in creating standardised language tests, ensuring their fairness, reliability, and validity as a tool for evaluating language proficiency.

By highlighting the positive impact of English language assessment on the employability of international students, this presentation aims to underscore the value of these tests in promoting global mobility and facilitating successful career transitions.

Bio
Dr Patrick Pheasant is a counsellor, coach and consultant in International Education and Training, specialising in English language teaching, learning and assessment. After 30 years in international education and training in Australia, USA, The Netherlands and Japan, and recently as CEO of NEAS Australia, Patrick supports the Global Partnerships & Recognitions Team at LanguageCert in the role of Head of Education Partnerships Australia & New Zealand.

Session 2A – Redesigning existing assessments with AI in mind
Room 5.61, Level 5
13:35 – 14:20 AEST

Paola Clews, ELT Teacher, The University of Sydney CET
Irma Basu, ELT Teacher, The University of Sydney CET
Chair: MC Rep

Abstract
The rise of AI has divided, worried, intrigued and excited the EAP community for the past two years, and with its growth expected to continue, many aspects of our university pathways courses need to reflect this new reality.

Informed by extensive research on best practice in curriculum design, including the currently under documented constraints and affordances posed by AI in our industry, this presentation will share with you the insights gained so far in the process of redesigning assessments for an EAP classroom in which students can and will use AI.

The presentation will revisit the principles that underpin assessment for learning, address where AI is placed in relation to our intended learning outcomes as a university pathway course and examine how to approach redesigning existing assessments. To better illustrate the process of redesigning assessment, we will be looking at a specific example of current writing and speaking assessments in the Direct Entry Course (DEC) and how these could be adapted into new tasks that keep the quality elements of the old along with carefully scaffolded elements of the new.

Bios
Paola Clews is an experienced ELT teacher. She holds a B.A in Modern Languages and Masters in Education and ICT from Macquarie University. She is passionate about helping students reach their goals through creating a supportive yet challenging classroom environment. She’s interested in Technology and its potential to support both teachers’ work and student learning. She works at CET at The University of Sydney.

Irma Basu has 20 years’ experience teaching English both in Australia and internationally. During this time, she has developed a keen interest in researching teacher identity and teacher emotions and how they play out both inside and outside of the classroom. Her most recent degree in Educational Psychology has fuelled her interest in exploring student motivation and the effects of cognitive load in the EAP classroom.

Session 2C – ‘Off the shelf’ AI: Purpose-built or Ill-fitting?
Room 5.30, Level 5
13:35 – 14:20 AEST

Jarrad Merlo, AI Product Manager, Pearson
Chair: MC Rep

Abstract
Artificial intelligence is advancing at a remarkable pace and is becoming increasingly user-friendly. Some tools appear perfectly suited for language testing purposes, making it tempting to assume that they can simply be used straight ‘off the shelf’. But is it really that straightforward? In this session, Jarrad Merlo, an AI Product Manager working in Pearson’s English Language Learning and Assessment Accelerator, will challenge this notion.

Drawing on his experience in developing digital learning and assessment products, he will advocate for research-backed and evidence-guided applications of new technologies. He will emphasise the importance of assumption testing during the “discovery” stage of product development. By adhering to a ‘principled’ approach, he believes we can significantly enhance the quality and equity of new language testing products.

Bio
Jarrad Merlo is an AI Product Manager working in Pearson’s English language learning and assessment accelerator. Jarrad also worked for OET designing digital-first workplace English proficiency tests. Prior to this, Jarrad spent ten years building E2Language, an online test preparation company that registered over 2 million IELTS, PTE, OET and TOEFL users and amassed over 3 million YouTube subscribers. Jarrad started his career teaching English in Indonesia, South Korea and Australia. His academic interests include workplace English language tests, automated speaking assessments, L1 transfer, English as a Lingua Franca and Indian English.

Afternoon Tea
Level 5
14:20 – 14:50 AEST

Refreshments & coffee/tea served.

Panel – State of Play: Does the Research Paper have a place in the Generative AI era?
The Auditorium, Level 2
14:50 – 15:40 AEST

Ruth Hellema, Assessment Design Specialist, Monash College
Dr Marzieh Sadeghpour, Program Leader, Monash College
Cara Dinneen, Education Manager, Macquarie University College
Brooke Donnelly, Academic Head, Embedded English Programs, UNSW College
Maria Naidoo, Academic Head, ELICOS Academic English, UNSW College
Chair: MC Rep

Abstract
Earlier this year at an English Australia Assessment SIG event, the continuation of the direct entry research paper was called into question. Challenges in upholding academic integrity in the face of Generative AI have made the use of the research paper, in its traditional form, unsustainable and unreliable as an assessment tool to develop and evaluate students’ academic skills. At that time, Monash, Macquarie and UNSW Colleges presented plans to modify or replace the research paper.

Now, six months on, changes are further advanced. In this session, the three colleges report their current state of play in direct entry program assessment, highlighting considerations, design principles and lessons learned along the way.

Bios
Ruth Hellema is an Assessment Design Specialist at Monash College. She has extensive experience in the design, development and delivery of a range of English language assessments, curriculum design and teaching, and has also held coordination roles. Her interests include authentic assessment, student feedback and exploring the potential of generative artificial intelligence in assessment design. Most recently, she has completed the Professional Certificate in Language Assessment at the University of Melbourne.

Dr Marzieh Sadeghpour holds a Ph.D. in Teaching English as an International Language from Monash University. With over 18 years of experience in English language teaching, she currently serves as the Program Leader of English at Monash College. Marzieh has conducted extensive research in the field of ELT, publishing in leading ELT journals. She is the author of “Englishes in English Language Teaching” (Routledge, 2019) and the editor of “Cultural Linguistics and World Englishes” (Springer, 2020)

Cara Dinneen is the Education Manager of English Language Programs at Macquarie University College. Cara holds a Master of TESOL, Grad Cert Educational Research, Grad Cert Business Educational Leadership, Trinity Diploma of TESOL, and a BA Communications. Cara has over 20 years’ experience in English language teaching, teacher training and leadership, having taught and managed programs in Australia, Oman and Spain. Cara has a strong background in teaching methodology and a creative, participant-focused approach to lesson design and teacher development. She also has a keen interest in learning and assessment design and is the Convenor of the English Australia Assessment Special Interest Group.

Brooke Donnelly is the Academic Head for Embedded English programs at UNSW College where she looks after delivery and quality in the Foundation, Diploma and Pre-masters programs. Brooke is responsible for supporting the team with implementing best practice in curriculum and assessment and steering the development of an embedded English support program across the Foundation and Diploma courses.

Maria Naidoo has managed ELICOS courses in Academic English Programs at UNSW College since 2017 and in this time has managed the University English Entry Courses (UEEC). Before this she worked as an ESL teacher, a resource writer and an Academic Manager in a range of contexts. Maria has been part of a range of English Language Support initiatives at UNSW, providing input on post-entry language support for international students in UNSW faculties. In 2021/2022, she supported the development of several intensive academic language workshops for the myEnglish Week initiative. Maria has co-convened the English Australia Direct Entry Program Special Interest Group since 2018.

Closing Remarks
Level 5
15:40 – 16:30 AEST

Social event. Refreshments served.

Registrations open on Monday, August 29

Register Here