2024

Tracing Success: A sector snapshot of Direct Entry ELICOS Tracer Studies in Australian Higher Education

Information for UECA Member Centres

Project Context

Direct entry (DE) programs in the Australian ELICOS (English Language Intensive Courses for Overseas Students) education sector are specialized courses designed to prepare international students for entry into Australian universities without requiring them to take additional standardised English proficiency tests such as IELTS or TOEFL (English Australia, 2017). These programs focus on developing students’ academic English skills and are designed to meet the demands of higher education (HE) studies. Upon successful completion, students are granted direct admission to partner institutions. It has been demonstrated DE programs can result in comparable or better academic outcomes for students in their chosen academic programs than for those admitted via meeting high-stakes English language test requirements (Roche, 2017).

The ELICOS Standards 2018 stipulate that for providers with a direct entry arrangement with a tertiary education institution, “formal measures must be in place to ensure that assessment outcomes are comparable to other criteria used for admission to the tertiary education course of study, or for admission to other similar courses of study” (Standard P4.1.C.ii). In order to meet this requirement, TEQSA suggests a number of possible approaches, including, external referencing through benchmarking, benchmarking against valid frameworks for language proficiency, external testing, and tracer studies of student cohorts (TEQSA, 2023).

Aligned with this guidance, University Language Centres employ a range of approaches to meet the requirements of Standard P4.1.C.ii, including the use of Tracer Studies. Tracer Studies broadly involve tracking DE English language pathway students’ performance in their degree studies to evaluate the extent to which direct entry programs have prepared students for their subsequent tertiary studies. In doing so, tracer studies seek to demonstrate that direct entry pathway assessment outcomes are comparable to the admission requirements for other students and methods of entry into tertiary institutions (TEQSA, 2023).

However, tracer studies are not well understood across the sector. There is no standard approach or format for conducting a Tracer Study, nor for the metrics assessed, the platforms used, reporting formats or the implications of the study outcomes. This not only presents a challenge for providers hoping to establish tracer studies at their own institutions, but also presents a challenge for the sector in establishing sector-wide best practices and presenting a cohesive picture of quality and reliability to regulatory bodies.

---

Project Scope

To help overcome these challenges, this project aims to build a picture of the current state of practice in conducting DE program Tracer Studies amongst UECA members. The output of the project will be a report which articulates which UECA member centres are conducting Tracer Studies and if so, the content of these studies, as well as the utility of their outcomes in informing improvements in direct entry programs. The report will identify the most common approaches to conducting Tracer Studies, with the advantages and challenges of several approaches highlighted. The report will provide several exemplar case studies which represent divergent or related practices, and, where appropriate, suggest recommendations for conducting Tracer Studies in the future. A commentary will also be included on the potential feasibility of completing cross-institutional reports on the outcomes of Tracer Studies and the potential correlations and metrics which would provide meaningful insight.

---

Out of Scope

No project can exhaustively cover a topic as broad as Tracer Studies. This project will not include commentary on the utility of university English requirements or means of testing, including IELTS and direct entry programs. It will not account for any TEQSA, ELICOS, or other regulations, policy directives, or regulatory requirements and the impact of these requirements on any offerings. Rather, it will contextualise Tracer Studies within the regulatory environment, and leave the specifics for institutions to elucidate depending on their individual needs and the emerging regulatory environment at the time. Lastly, this project will not create a journal-ready or publication-ready report in a format which could be submitted to a peer-reviewed publication, although this may be a future opportunity if the outcomes and recommendations listed in the project report are deemed of interest to the wider community.

---

Project Methodology

UECA members (n ≈ 28) will be provided with questionnaire via Qualtrics which asks them whether they conduct Tracer Studies. Those that do conduct these Studies will be requested to provide further information on their studies: what they measure, platforms used, reporting formats and the utility of the results. From those members who indicate that they are comfortable to be contacted for follow-up conversations, a selection (n ≤ 5) will be invited to participate in subsequent interviews, which will be used to elucidate a more detailed understanding of their approach to Tracer Studies. Together, the questionnaire and interview data will be coded using content analysis, summarised, and analysed to develop case studies which demonstrate divergent and related practices in conducting Tracer Studies, and to develop a list of recommendations for conducting Tracer Studies in the future.

A Steering Committee of UECA Committee members will provide direction, oversight, and decision-making support for the project, ensuring alignment with UECA’s goals and objectives. They will assist in managing project risks, resolving issues, and facilitating communication among stakeholders to guide the project towards successful completion.

---

Project Outputs

The outputs for this project will include:

• a questionnaire, the data from which will be coded and summarised.
• follow-up interviews with key providers who provide their consent for further engagement will be conducted, with the data from these again coded and summarised.
• a report of approximately 25 pages will then be compiled and presented to the UECA Committee for their comment, before being presented to the wider UECA community.

---

Detailed outline of project activities: Activities and Outcomes

Background research on tracer studies

Questionnaire (instrument, building, committee review and edits, stakeholder consultation and continued engagement)

Analysis of survey data in qualitative and quantitative context as well as creation of appropriate visualisations

Presentation of survey data to UECA committee, discussion and feedback

Conducting, coding, and analysing follow-up interviews

Presentation of interview data to UECA committee, discussion and feedback

Preparation of case studies, including consultation with relevant stakeholders to deep-dive into relevant examples and processes which will underpin case study

Preparation of final draft report, including presentation to UECA committee, discussion and feedback

Completion of final report

---

Citations

Commonwealth of Australia. (2017). ELICOS Standards 2018. Available at: https://www.legislation.gov.au/F2017L01349/latest/text [Accessed 19 Feb. 2024].

English Australia (2017). Guide to Best Practice in ‘Direct Entry’ Programs (2017) Summary. https://www.englishaustralia.com.au/documents/item/195

Roche, T. (2017). Assessing the role of digital literacy in English for Academic Purposes university pathway programs. Journal of Academic Language and Learning, 11(1), A71-A87. Retrieved from https://journal.aall.org.au/index.php/jall/article/view/439

TEQSA. (2023). ELICOS direct entry guide. [online] Available at: https://www.teqsa.gov.au/guides

---

2022

UECA Integrated Assessment Grant Opportunity

Grants of up to $8000 will be provided for up to 5 projects

University English Centres Australia (UECA) is committed to supporting members in the provision of valid and reliable assessment of their students’ language learning outcomes.

With this end in mind, UECA is offering up to five (5) grants to support projects that further our understanding of integrated assessments and/or support Centres’ capacity to deliver quality-integrated assessments.

While the initial impact might be at the level of the institution, there should be clear potential value to the broader UECA membership or sector as a whole.

Full details here

The Opportunity
Grants of up to $8000 will be provided for up to 5 projects which meet the criteria set out below.

The funds may be used across participating Centres to:

• buy out staff time to enable them to spend time on the project;
• provide administrative support;
• buy in specialist support, such as a digital/educational designer;
• purchase required materials for the project;
• cover other expenses for activities which would assist in the successful delivery of the project.

Grants will normally be disbursed on completion of the project and provision of evidence of expenses incurred.

Full details here

Important Dates

• Monday 23 May 2022
  Grant applications open
• Tuesday 21 June
  Virtual meeting to discuss the grant criteria and raise any questions
• Monday 01 August 2022
  Due date for submission of bids
• Tuesday 30 August 2022
  Successful grant applicants announced

Full Grant description and application details here

If you have any questions about this Grant Opportunity and the bidding process, please contact Simon Winetroube – simon.winetroube@curtin.edu.au

---

2019 – 2022

The UECA Innovation and Leadership Award

University English Centres Australia is committed to facilitating exceptional student experience through excellence in teaching, ongoing professional development and also building research capacity in the global TESOL community. The UECA Leadership and Innovation award generously funded by Education Solutions Australasia. The topic and focus of the award are decided annually by the UECA Committee.

2022 award details coming soon.

---

2018

UECA Benchmarking Project

UECA has supported its members through a range of benchmarking projects where Centres shared and compared data on a range of operational aspects, such as, enrolment details, staffing profiles and marketing practices.

In 2018, it was agreed to focus benchmarking activities on Centre policies, processes and assessment standards.

With this in mind, UECA commissioned the External Referencing of the ELICOS Standards and International Education [ERESIE] Project with Dr Sara Booth and Associate Professor Thomas Roche appointed as project leads.

Members agreed that the primary focus of the initial national benchmarking project would be the newly updated ELICOS (2018) Standard P4: Assessment of ELICOS students with a specific focus on written assessment.

The project was designed with a view to seeking peer feedback on Centres’ ELICOS Direct Entry program learning outcomes, giving peers a mechanism to provide an assessment of program standards, confirmation of good practice and suggested areas for improvement.

The Benchmarking report, available for download below, reiterates and consolidates findings from the extensive UECA National External Referencing Project Assessment Policies and Processes Report. The current document includes additional discussion of those findings for a wider audience.

UECA would like to formally acknowledge and thank the author of that original, comprehensive report, Dr Sara Booth.

The UECA Committee would like to acknowledge the good faith with which participating Centres have engaged in this project as they reviewed the ELICOS Direct Entry programs of other Centres and responded to the feedback they received on their own.

The project involved time commitment not only from Centre managers, but also program coordinators, lecturers, and teachers. We would like to thank them for their contribution to this project, the results of which are truly shared by all members.

Finally, the Committee would like to acknowledge the leadership of Thomas Roche and Sara Booth who through the benchmarking project have lead innovation in the ELICOS and higher education pathway sectors in Australia, in creating a robust mechanism and rigorous processes for ensuring the comparability of the standards of our programs.

You can download the UECA Benchmarking Project Report here.

Participating Centres