ALTAANZ
  • About ALTAANZ
  • ALTAANZ Committee
    • Current Committee
    • 2021 Committee
    • 2020 ALTAANZ Committee
    • 2018 - 2019 ALTAANZ Committee
    • 2017 ALTAANZ Committee
    • 2016 ALTAANZ Committee
    • 2015 ALTAANZ Committee
    • 2014 ALTAANZ Committee
    • 2013 ALTAANZ Committee
    • 2012 ALTAANZ Committee
    • 2011 ALTAANZ Committee
  • Joining ALTAANZ
  • ALANZ - ALAA - ALTAANZ 2022
  • ALTAANZ Online Research Forum 2021
  • LTRC/ALTAANZ Online Celebratory event 2020
    • About the event
    • Event Programme
    • LTRC Anniversary Symposium
  • Past ALTAANZ Conferences
    • ALANZ / ALAA / ALTAANZ AUCKLAND 2017
    • ALTAANZ Conference Auckland 2016 >
      • Keynote Speakers >
        • Plenary Abstracts
      • Teachers' Day
      • Pre-conference workshops
      • Conference programme
    • ALTAANZ Conference Brisbane 2014
    • ALTAANZ Conference Sydney 2012
  • Past ALTAANZ / LTRC Workshops
    • LTRC / ALTAANZ Workshops July 2014 >
      • Test analysis for teachers
      • Diagnostic assessment in the language classroom
      • Responding to student writing
      • Assessing Pragmatics
      • Introduction to Rasch measurement
      • Introduction to many-facet Rasch measurement
    • LTRC / ALTAANZ workshops September 2015 >
      • A Practical Approach to Questionnaire Construction for Language Assessment Research
      • Integrating self- and peer-assessment into the language classroom
      • Implementing and assessing collaborative writing activities
      • Assessing Vocabulary
      • Revisiting language constructs
  • Studies in Language Assessment (formerly PLTA)
    • Early View Articles
    • Current Issue
    • About SiLA
    • Past Issues >
      • 2022
      • 2021
      • 2020
      • 2019
      • 2018
      • 2017
      • 2016
      • 2015
      • 2014
      • 2013
      • 2012
    • Editorial Board
    • Contributors
  • ALTAANZ Awards
    • ALTAANZ Best Student Paper Award
    • Penny McKay Award
    • PLTA Best Paper Award
  • Sponsorship for Educational Activities
  • Newsletter: Language Assessment Matters
  • Contact us
Examination of CEFR-J spoken interaction tasks using many-facet Rasch measurement and generalizability theory
Rie Koizumi, Juntendo University, Chiba
Emiko Kaneko, University of Aizu, Fukushima, Japan
Eric Setoguchi, University of California, Los Angeles, USA
Yo In’nami, Chuo University, Tokyo, Japan
Naoyuki Naganuma, Tokai University, Kanagawa, Japan
https://doi/10.0000/0000
Volume 8, Issue 2, 2019
Abstract: Attempts are underway to develop prototype tasks, based on a Japanese version of the Common European Framework of Reference for Languages (CEFR; Council of Europe, 2001; CEFR-J; Negishi, Takada, & Tono, 2013). As part of this larger project, the current paper reports on the creation of spoken interaction tasks for five levels (Pre-A1, A1.1, A1.2, A1.3, and A2.1). Tasks were undertaken by 66 Japanese university students. Two raters evaluated their interactions using a three-level holistic rating scale, and 20% of the performances were double rated. The spoken ratings were analysed using many-facet Rasch measurement (MFRM) and generalizability theory (G-theory). MFRM showed that all the tasks fit the Rasch model well, the scale functioned satisfactorily, and the difficulty of the tasks generally concurred with CEFR-J levels. Results from G-theory that employed the p x t design, including tasks as a facet, showed the different proportion of variance accounted for by tasks, as well as the number of tasks that could be required to ensure sufficiently high reliability. The MFRM and Gtheory results effectively revealed areas for improving spoken interaction tasks; the results also showed the usefulness of combining the two methods for task development and revision.
Keywords: CEFR-J, generalizability theory, many-facet Rasch measurement, spoken interaction
Click to download Full Text