Revisiting language constructs – Recent theoretical advances and applications to practical test design, development and validation
Xiaoming Xi, Educational Testing Service
While theoretical approaches to defining language constructs continue to be debated in the last two decades, more recent conceptualizations have converged on the interactionalist view, which highlights the important role of language use context (Chapelle, 1998; Chalhoub-Deville, 2004; Bachman, 2007). This interactionalist approach has a strong potential to build on the strengths of two prevalent, yet seemingly contrasting approaches to test design and validation: task-based (Norris, 2002) and construct-based (Bachman, 2002). However, despite the theoretical advances, the practical implications of the interactionalist approach remain to be explored. This workshop will focus on explicating the interactionalist approach for the purpose of guiding the practical design, development and validation of tests using speaking as an example.
In alignment with the interactionalist view, I argue for a multi-faceted approach to construct definition, consisting of two key elements: foundational and higher-order language skills and use contexts. Theoretical language ability models focus on delineating the foundational and higher-order knowledge and skills called for in language use activities (Canale & Swain, 1980; Bachman, 1990; Bachman & Palmer, 1996; 2010; Fulcher, 2003). Frameworks for language use have attempted to describe personal attributes of examinees and characteristics of the language use task and situation that shape language use (Bachman & Palmer, 2010). Characteristics of the language use task and situation essentially define the context of language use.
Prevalent theoretical models attempt to articulate the language knowledge components called upon in language use activities in all modalities to facilitate generalization across them. I propose a model of oral communicative competence to provide an organizing structure for describing all underlying components relevant to oral communication, and to provide opportunities for highlighting salient components in this modality-specific model.
I describe a scheme to characterize key contextual factors of oral communication as layers, each of which adds specificity to constrain the contextual parameters of tasks. Test designers need to prioritize the inclusion of key facets of the language use context that characterize the target language use domain most meaningfully, and that are expected to account for variation in the quality and features of examinees’ responses.
I will use examples to illustrate how the interactionalist approach could be applied to drive optimal test design and development and to guide validation. Given the primary concern about the generalizability of highly contextualized tasks, I will discuss how task and form comparability can be supported. I will also discuss the roles that construct-based and task-based approaches play in the assessment process and their relative strengths in this integrated interactional approach to construct definition.
Biography:
Xiaoming Xi is Senior Director of the Research Center for English Language Learning and Assessment in the Educational Testing Service in the United States. She has published widely in areas including validity and fairness issues in the broader context of test use, validity frameworks for automated scoring, automated scoring of speech, and task design, scoring and rater issues in speaking assessment. She is a recipient of the 2005 International Language Testing Association (ILTA) Best Language Testing Paper Award, and currently serves on the Editorial Boards of Language Testing and Language Assessment Quarterly. She received her Ph.D. in second/foreign language assessment from the University of California, Los Angeles.
Xiaoming Xi, Educational Testing Service
While theoretical approaches to defining language constructs continue to be debated in the last two decades, more recent conceptualizations have converged on the interactionalist view, which highlights the important role of language use context (Chapelle, 1998; Chalhoub-Deville, 2004; Bachman, 2007). This interactionalist approach has a strong potential to build on the strengths of two prevalent, yet seemingly contrasting approaches to test design and validation: task-based (Norris, 2002) and construct-based (Bachman, 2002). However, despite the theoretical advances, the practical implications of the interactionalist approach remain to be explored. This workshop will focus on explicating the interactionalist approach for the purpose of guiding the practical design, development and validation of tests using speaking as an example.
In alignment with the interactionalist view, I argue for a multi-faceted approach to construct definition, consisting of two key elements: foundational and higher-order language skills and use contexts. Theoretical language ability models focus on delineating the foundational and higher-order knowledge and skills called for in language use activities (Canale & Swain, 1980; Bachman, 1990; Bachman & Palmer, 1996; 2010; Fulcher, 2003). Frameworks for language use have attempted to describe personal attributes of examinees and characteristics of the language use task and situation that shape language use (Bachman & Palmer, 2010). Characteristics of the language use task and situation essentially define the context of language use.
Prevalent theoretical models attempt to articulate the language knowledge components called upon in language use activities in all modalities to facilitate generalization across them. I propose a model of oral communicative competence to provide an organizing structure for describing all underlying components relevant to oral communication, and to provide opportunities for highlighting salient components in this modality-specific model.
I describe a scheme to characterize key contextual factors of oral communication as layers, each of which adds specificity to constrain the contextual parameters of tasks. Test designers need to prioritize the inclusion of key facets of the language use context that characterize the target language use domain most meaningfully, and that are expected to account for variation in the quality and features of examinees’ responses.
I will use examples to illustrate how the interactionalist approach could be applied to drive optimal test design and development and to guide validation. Given the primary concern about the generalizability of highly contextualized tasks, I will discuss how task and form comparability can be supported. I will also discuss the roles that construct-based and task-based approaches play in the assessment process and their relative strengths in this integrated interactional approach to construct definition.
Biography:
Xiaoming Xi is Senior Director of the Research Center for English Language Learning and Assessment in the Educational Testing Service in the United States. She has published widely in areas including validity and fairness issues in the broader context of test use, validity frameworks for automated scoring, automated scoring of speech, and task design, scoring and rater issues in speaking assessment. She is a recipient of the 2005 International Language Testing Association (ILTA) Best Language Testing Paper Award, and currently serves on the Editorial Boards of Language Testing and Language Assessment Quarterly. She received her Ph.D. in second/foreign language assessment from the University of California, Los Angeles.