Investigating cognitive processes in different item formats in reading tests through eye-tracking and verbal protocols
Hatice Akgün, School of Foreign Languages, Marmara University, Istanbul, Turkey
Aylin Ünaldı, School of Education and Professional Development, University of Huddersfield, Huddersfield, UK
https://doi/10.0000/0000
Volume 11, Issue 2, 2022
Abstract: This study has investigated the differences in cognitive processes that test-takers undergo while answering reading comprehension questions in multiple-choice and open-ended short answer formats. For this purpose, data were collected from a group of undergraduate students in an English medium university through eye-tracking technology, immediate retrospective verbal protocols, and short semi-structured interviews. The results showed that the participants used careful reading skills more and comprehended the text more thoroughly in the open-ended format. However, in the MC format, they read less carefully and used more test-taking strategies. These findings contribute to the ongoing discussion on how item format can alter the cognitive processes in a reading comprehension test and confirm the effectiveness of eye-tracking in unveiling cognitive processes in combination with qualitative methods. This study has implications for reading test development.
Keywords: Eye-tracking in reading, reading processes, item format effect in reading tests, cognitive validity in reading tests