Samuel Messick said, "Inferences are hypotheses, and the validation of inferences is hypothesis testing.However, it is not hypothesis testing in isolation but, rather, theory testing more generally because the source, meaning, and import of score-based hypotheses derive from the interpretive theories of score meaning in which these hypotheses are rooted.As a consequence, test validation isbasically both theory-driven and data-driven" (1990).

Messick 1990-"But a half dozen or so categories of the following sort provide a workable level for highlighting similarities and differences among validation approaches:

1. Appraise the relevance and representativeness of the test content in relation to the content of the behavioral or performance domain about which inferences are to be drawn orpredictions made.

2. Examine relationships among responses to the tasks, items, or parts of the test -- that is, delineate the internal structureof test responses.

3. Survey relationships of the test scores with other measuresand background variables -- that is, elaborate the test's external structure.

4. Directly probe the ways in which individuals cope with the items or tasks, in an effort to illuminatethe processes underlying item response ane, task performance.

5. Investigate uniformities and differences in these test processes and structures over time or across groups and settings --that is, ascertain that the generalizability (and limits) of test interpretation and use are appropriate to the construct andcontexts at issue.

6. Evaluate the degree to which tesl: scores display appropriate or theoretically expected variations as a function of instructional and other interventions or as a result of experimental manipulation of content and conditions.

7. Appraise the value implications and social consequences of interpreting and using the test scores in the proposed ways, scrutinizing not only the intended outcomes but also unintendedside effects -- in particular, evaluate the extent to which (or,preferably, discount the possibility that) any adverse consequences of testing derive from sources of score invaliditysuch as irrelevant test variance."

Validation is the process of developing and evaluating evidence for a proposed score interpretation and use. The argument-based approach to test validation is useful for some large-scale assessments. Further refinement is necessary for broader use by test developers and practitioners. 

 Argument Based Validation

Chapelle, C. A. (2020). Argument-based validation in testing and assessment. Thousand Oaks, CA: Sage Publishing.

Chapelle, C. A., Enright, M. & Jamieson, J. (Eds.) (2008). Building a validity argument for the Test of English as a Foreign Language. London: Routledge.

What are the stages in argument-based validation?

How do you categorize claims in an IUA?

How do you structure the claims in the arguments?

What evidence do we collect?