Intelligibility and comprehensibility across Aptis Speaking tasks
Daniel R. Isbell, University of Hawai’i at Mānoa, USA
Dustin Crowther, University of Hawai’i at Mānoa, USA
Jieun Kim, University of Hawai’i at Mānoa, USA / Soongsil University, Republic of Korea
Yoonseo Kim, University of Hawai’i at Mānoa, USA
Dustin Crowther, University of Hawai’i at Mānoa, USA
Jieun Kim, University of Hawai’i at Mānoa, USA / Soongsil University, Republic of Korea
Yoonseo Kim, University of Hawai’i at Mānoa, USA
|
https://doi.org/10.58379/YWIO8217
|
Volume 14, Issue 1, 2025
|
Abstract: Current Aptis Speaking rubrics reflect Common European Framework of Reference (CEFR) Phonological Control descriptors, which were revised in 2018 to better emphasize the intelligibility of second language (L2) speech. The present study investigates the validity of Aptis Speaking scores and related CEFR-based interpretations by comparing official Aptis Speaking scores to laypersons’ assessments of intelligibility, a measure of accuracy of understanding using orthographic transcription (0-100%), and comprehensibility, a measure of ease of understanding using ratings on a scale of 1 to 9. Additionally, as Aptis Speaking tasks feature several target performance levels, we considered relationships between task complexity and speakers’ intelligibility and comprehensibility. Archived speaking performances from 50 Aptis examinees were assessed by layperson listeners for intelligibility (n = 562) and comprehensibility (n = 567). Comprehensibility was generally a stronger predictor of Aptis Speaking scores than intelligibility for both overall and task-level scores. Segmented regressions revealed specific breakpoints in which predictive power was greatest for each dimension (intelligibility up to 70% in transcription accuracy; comprehensibility ≥ 2). In sum, results from the current study generally provided support for the current Aptis Speaking rubrics, though the way in which comprehensibility is described at the upper levels may benefit from more nuance.
Keywords: Aptis, CEFR, comprehensibility, intelligibility, layperson listener