LTRC anniversary symposium, thursday 19 november, 3.30pm AEDT
You can find the recording of this event at the University of Melbourne's website.
Click here to link through to the video.
Click here to link through to the video.
Mapping tensions between language testing, policies and practices
Language tests are widely used in Australia and elsewhere to implement institutional policy, to serve educational goals and to regulate transitions into study and employment. The different roles testing activities can serve in these contexts and the fairness and justice implications of using or misusing language tests for such consequential purposes have been the focus of growing attention in recent years. Accordingly, the need to more effectively engage with policy makers, employers, educators, language learners and test takers to better understand these implications has become ever more pressing. A series of 10-minute presentations by staff and associates of the Language Testing Research Centre (see titles and authors below) will reflect on how language testers engage with policy, the different ways in which test constructs mediate policy, the conflict between policy imperatives and test developers’ intentions, and how test scores are interpreted and reshaped in practice by different test users. Two discussants and founding fathers of the Centre, Professors Joe Lo Bianco and Tim McNamara, will respond to these presentations and consider their contribution to current debates in the fields of language assessment, language policy and applied linguistics more generally.
Framing the LTRC’s policy contribution in the languages arena
Cathie Elder
The LTRC, as a self-funding Centre, responds by necessity to policy shifts, whether these occur at broader societal or more local institutional and program levels. Its language testing activities also have the potential to influence policy, whether directly or indirectly. Focussing on examples of the Centre’s work in languages education, this talk considers the diverse orientations adopted (i.e. to inform, enact or evaluate policy). I speculate about the policy impact in each case, highlighting complexities encountered and concluding with recommendations for policy responsible language testing.
The LTRC, as a self-funding Centre, responds by necessity to policy shifts, whether these occur at broader societal or more local institutional and program levels. Its language testing activities also have the potential to influence policy, whether directly or indirectly. Focussing on examples of the Centre’s work in languages education, this talk considers the diverse orientations adopted (i.e. to inform, enact or evaluate policy). I speculate about the policy impact in each case, highlighting complexities encountered and concluding with recommendations for policy responsible language testing.
the challenges of providing expert advice in policy contexts
Ute Knoch
Language testers can have various roles in relation to the use of language tests and policy. One such role may be to provide expert advice in policy formation or policy review contexts. Such instances are often not documented systematically, as advice may be provided in informal or closed meetings, or confidential documents, which are not available to the public. The challenges associated with providing advice are also rarely recorded. In this paper, I describe three instances in which the Language Testing Research Centre was invited to provide external policy advice. I specifically reflect on how we came to be invited to provide advice, what advice we were asked to provide, the complexity of providing advice in each instance, and whether the advice was taken up by the policy makers. The paper concludes with implications for training new language testers.
Language testers can have various roles in relation to the use of language tests and policy. One such role may be to provide expert advice in policy formation or policy review contexts. Such instances are often not documented systematically, as advice may be provided in informal or closed meetings, or confidential documents, which are not available to the public. The challenges associated with providing advice are also rarely recorded. In this paper, I describe three instances in which the Language Testing Research Centre was invited to provide external policy advice. I specifically reflect on how we came to be invited to provide advice, what advice we were asked to provide, the complexity of providing advice in each instance, and whether the advice was taken up by the policy makers. The paper concludes with implications for training new language testers.
navigating tensions between language testing intentions and policy imperatives: the case of the college english test (cet) in China
Jason Fan & Yan Jin
In this talk, we demonstrate the conflicts or tensions between the intended uses of a language test and the actual uses, misuses and even abuses of the test in the real world, driven by various policy imperatives. We use the College English Test (CET), an English proficiency test targeting non-English major undergraduates in China’s universities, as an example. Our analysis of the CET was guided by Knoch and Elder’s (2013) framework for evaluating post-entry language assessment and revealed that the CET has been used for several purposes unintended by the test developer both in the educational community and the social domain, due to different policy imperatives. To address such issues, we argue that continuous professionalisation of language testing is instrumental in building an ethical milieu, which in turn can help to promote more responsible uses of test results.
In this talk, we demonstrate the conflicts or tensions between the intended uses of a language test and the actual uses, misuses and even abuses of the test in the real world, driven by various policy imperatives. We use the College English Test (CET), an English proficiency test targeting non-English major undergraduates in China’s universities, as an example. Our analysis of the CET was guided by Knoch and Elder’s (2013) framework for evaluating post-entry language assessment and revealed that the CET has been used for several purposes unintended by the test developer both in the educational community and the social domain, due to different policy imperatives. To address such issues, we argue that continuous professionalisation of language testing is instrumental in building an ethical milieu, which in turn can help to promote more responsible uses of test results.
negotiating the boundaries of english: the role of tests and language testers in australia's skilled migration policy space
Kellie Frost
While the importance of accounting for the use of tests as policy instruments is by now widely acknowledged, validation and fairness frameworks in language testing rest on core assumptions which, I argue, are often incommensurate with the ways in which policy intentions are articulated and pursued. This is especially the case in the context of Australia’s skilled migration policy, where tests are embedded within a range of selection processes across various intersecting layers of policy, each determined by different stakeholders with different needs and intentions. In this paper, I will discuss the roles tests play in this complex policy setting, from policy maker perspectives, and from migrant perspectives as they respond to their entangled test-in-policy experiences. Highlighting the inevitable ‘validity chaos’ this policy context engenders for language testers, I conclude by arguing for a renewed criticality in language testing, where we expand our lens beyond evaluations of how well, if at all, test uses align with notions of best and ethical practice in our field, to engage with the wider discursive space within which problems of language and of policy are imagined, and to interrogate the ways in which we, as a discipline, are part of a material-discursive apparatus that produces idealised language users, workers, and citizens, together with the various inclusions and exclusions this entails.
While the importance of accounting for the use of tests as policy instruments is by now widely acknowledged, validation and fairness frameworks in language testing rest on core assumptions which, I argue, are often incommensurate with the ways in which policy intentions are articulated and pursued. This is especially the case in the context of Australia’s skilled migration policy, where tests are embedded within a range of selection processes across various intersecting layers of policy, each determined by different stakeholders with different needs and intentions. In this paper, I will discuss the roles tests play in this complex policy setting, from policy maker perspectives, and from migrant perspectives as they respond to their entangled test-in-policy experiences. Highlighting the inevitable ‘validity chaos’ this policy context engenders for language testers, I conclude by arguing for a renewed criticality in language testing, where we expand our lens beyond evaluations of how well, if at all, test uses align with notions of best and ethical practice in our field, to engage with the wider discursive space within which problems of language and of policy are imagined, and to interrogate the ways in which we, as a discipline, are part of a material-discursive apparatus that produces idealised language users, workers, and citizens, together with the various inclusions and exclusions this entails.
discussants
Tim McNamara & Joe Lo Bianco