ALTAANZ
  • About ALTAANZ
  • ALTAANZ Committee
    • Current Committee
    • 2021 Committee
    • 2020 ALTAANZ Committee
    • 2018 - 2019 ALTAANZ Committee
    • 2017 ALTAANZ Committee
    • 2016 ALTAANZ Committee
    • 2015 ALTAANZ Committee
    • 2014 ALTAANZ Committee
    • 2013 ALTAANZ Committee
    • 2012 ALTAANZ Committee
    • 2011 ALTAANZ Committee
  • Joining ALTAANZ
  • ALTAANZ Online Conference 2023: Call for Papers
  • Past ALTAANZ Conferences
    • ALTAANZ Online Research Forum 2021
    • ALANZ - ALAA - ALTAANZ 2022
    • LTRC/ALTAANZ Online Celebratory event 2020 >
      • About the event
      • Event Programme
      • LTRC Anniversary Symposium
    • ALANZ / ALAA / ALTAANZ AUCKLAND 2017
    • ALTAANZ Conference Auckland 2016 >
      • Keynote Speakers >
        • Plenary Abstracts
      • Teachers' Day
      • Pre-conference workshops
      • Conference programme
    • ALTAANZ Conference Brisbane 2014
    • ALTAANZ Conference Sydney 2012
  • Past ALTAANZ / LTRC Workshops
    • LTRC / ALTAANZ Workshops July 2014 >
      • Test analysis for teachers
      • Diagnostic assessment in the language classroom
      • Responding to student writing
      • Assessing Pragmatics
      • Introduction to Rasch measurement
      • Introduction to many-facet Rasch measurement
    • LTRC / ALTAANZ workshops September 2015 >
      • A Practical Approach to Questionnaire Construction for Language Assessment Research
      • Integrating self- and peer-assessment into the language classroom
      • Implementing and assessing collaborative writing activities
      • Assessing Vocabulary
      • Revisiting language constructs
  • Studies in Language Assessment (formerly PLTA)
    • Current Issue
    • About SiLA
    • Past Issues >
      • 2022
      • 2021
      • 2020
      • 2019
      • 2018
      • 2017
      • 2016
      • 2015
      • 2014
      • 2013
      • 2012
    • Editorial Board
    • Contributors
  • ALTAANZ Awards
    • ALTAANZ Best Student Paper Award
    • Penny McKay Award
    • PLTA Best Paper Award
  • Sponsorship for Educational Activities
  • Newsletter: Language Assessment Matters
  • Contact us
Using corpus complexity analyses to refine a holistic ESL writing placement rubric
Jeremy R. Gevara, Pennsylvania State University
https://doi/10.0000/0000
Volume 4, Issue 2, 2015
Abstract: The purpose of this study is to determine if corpus analysis tools can identify linguistic features within writing placement samples that are significantly different between levels within a higher education language program. Although commercial tests are widely used for placement decisions, local performance assessments have become more common compliments that better adhere to communicative language teaching. At the university where this study was conducted, raters use a holistic rubric to score students’ responses to one academic topic. The scoring process is fast when rates agree but too time consuming when raters search for information to resolve disagreements. Writing placement samples from 123 former students’ essays at an Intensive English Program were used to compile a corpus. I divided the writing samples into four folders that correspond with the program levels and analyzed the folders using syntactic, lexical, and essay complexity analyzers. I utilized the robustness of the ANOVA to account for assumption violations. Data that violated the normality assumption were first analyzed using the Kruskal-Wallis Test. Those variables showing significant differences between levels were then analyzed using ANOVA and the appropriate post-hoc tests. Results show significant between group differences with lexical and word types and tokens, complex nominal, verb phrases, and ideas. I discuss the interpretation of these variables as well as show how administrators used this information to revise the rubric from Version I to Version II. Broader implications from this study are the use of corpus research tools to operationalize performance for the purposes of model building. 
Keywords: Writing, Placement, Holistic, Rubric, Corpus
Click to download Full Text