Validity argument of an Interactional Competence checklist: Perspectives and many-facets Rasch analysis
Zahra Montasseri & Alireza Ahmadi, Shiraz University, Iran
|
https://doi.org/10.58379/MEZB3039
|
Volume 13, Issue 1, 2024
|
Abstract: The continued scrutiny of interactional competence (IC) as a sociolinguistically-based construct is gaining momentum in L2 assessment. Previous contributions have emphasized the multi-faceted nature of IC in operationalization and assessment. Among the few assessment tools developed for IC, May et al.’s (2020) checklist is one of the most comprehensive. This study evaluated the decision inference of this IC checklist based on Knoch and Chapelle’s (2017) argument-based approach to validation. Several sources of data, including test score data, semi-structured interviews, and expert reviews were used to investigate the proposed assumptions. Four raters evaluated 62 upper-intermediate and advanced test-takers’ paired and group oral performances using the adapted IC checklist. Furthermore, five experts in the field of language assessment and 10 language teachers participated in the study. To analyze the data, MFRM analysis through FACETS and thematic analysis were employed. The results provided partial evidence for the assumptions, thus leading to an overall partial support to the decision inference. The IC checklist showed to be a valid instrument for differentiating between test-takers at different IC levels, interpreting scores, and informing teaching and learning. The major drawback of the checklist was associated with a lack of descriptive levels and numerical values.
Keywords: interactional competence, argument-based validation, oral performance assessment, low-stake tests, learning-oriented assessment