Published online by Cambridge University Press: 14 January 2021
This chapter presents argument-based validation research to evaluate the interpretation of scores from an English collocational ability test. The argument-based validity framework guided the development of an interpretation/use argument that helped identify the types and amount of research needed to evaluate the plausibility of the claims about test score interpretation. Research presented in this chapter focuses on the explanation inference which is made when test users interpret the score as having substantive meaning about the construct assessed, specifically the construct of collocational ability in academic writing. The construct was defined by specifying the nature and scope of the construct following an interactionalist construct, which consists of three parts 1) the knowledge skills and abilities of a trait, 2) the types of contexts that delimit the scope of applicability of the trait, and 3) the metacognitive strategies to put the trait into use in those contexts. The target collocation was identified and defined based on applied linguistics theory and research, analysis of test-takers‘ responses to items on the test and statistical analysis of test scores. The relationship between collocational ability and other constructs of language ability were hypothesized in the nomological network to provide a basis for interpreting observed statistical relationships among sets of test scores reflecting those constructs. Evidence from screen capturing analysis, responses on a post-test survey and post-test interviews provided backing of strategy use. Data were collected in two phases of an embedded, sequential explanatory design to first obtain results from qualitative analysis of test takers' responses and then explain the results of the quantitative data with results from the supplementary qualitative data. Evidence collected in this study supported the construct of collocational ability underlying the explanation inference and demonstrated how argument-based validity can be used to lay a foundation for interpretation of test scores that is essential to score meaning.