jeromedelisle.org

Describe and detail an evaluation programme that closely follows an argument-based validation methodology for any named educational test, assessment system, or psychological instrument.

1. Identify with evidence the stated or implied claims and uses and classify these according to the category of inference

2. Suggest a research agenda based upon a theoretical analysis of the plausibility and coherence of the explicated IUA following Kane (2013)

3. Detail and analyze the existing empirical evidence on each category of inference and each claim- follow Chapelle (2020)

4. Make an integrated judgment on interpretations and uses based upon the evidence analyzed

5. Describe additional evidence to be collected in further validation studies.

Scoring Scheme (15 marks each)

 

---

Fairness

You are the Minister of Education for Trinidad and Tobago. Write a speech exploring the different philosophical perspectives, legal and social constraints, and technical requirements for ensuring greater fairness in current 11+, 16+, and 18+ selection systems. Consider the issue of moving towards justice-oriented rather than efficiency-oriented assessments.

1. Consider multiple philosophical perspectives

2. Consider legal and social constraints

3. Consider technical requirements related to fairness and validity

4. Considers all three selection/placement systems

5. Explores and evaluates the concept of justice-oriented assessments

Scoring Scheme (15 marks each)



---

What is a justice-oriented testing system?

Zachary Stein writes in his Harvard thesis (2014, p. 22-23)

Justice-oriented testing infrastructures, on the other hand, are built to assure that objectivity and efficiency are achieved, but not at the expense of being relevant and beneficial to those most affected by them. A testing infrastructure that honors the metrological rights of students and teachers is more likely to actualize the commitment of an educational system to provide for the full range of educational primary goods. It contributes to a system of background institutions that promote equality of opportunity and facilitate the self-actualization of all students. Needless to say, there has never been a pure instance of justice-oriented testing (just as there has never been a pure instance of efficiency-oriented testing). But attempts at justice-oriented testing have been in evidence since the first testing infrastructures were built in the early decades of the 20th century. Testing infrastructures have served a variety of functions that are a necessary part of any educational system committed to social justice, including administering a kind of pure procedural justice in the allocation of opportunity, assuring the equitable distribution of educational primary goods, and identifying the unique learning needs of each student.

 

Fair assessment viewed through the lenses of measurement theory

by Isabel Nisbet & Stuart D Shaw

Nisbet, I., & Shaw, S. D. (2019). Fair assessment viewed through the lenses of measurement theory. Assessment in Education: Principles, Policy & Practice, 26(5), 612-629.

The first is a formal sense, denoting accuracy or appropriate application of a rule or design. If I breach a formal rule of a game it is fair for the referee to blow the whistle. In this formal sense there is no necessary implication that the rule in question is good – just that it has been applied correctly.

The second sense may be labelled an implied contractual sense: something is fair if it meets the legitimate expectations of those affected. This was one of the meanings of ‘fairness’ in the UK court case brought in 2013 against Ofqual (the exams regulator in England) and others, following disappointment by some recipients of their GCSE English grades in summer 2012. The judgement referred to many aspects of unfairness in the context of the case and included a helpful account of ‘legitimate expectation’:

“A legitimate expectation may arise either out of an express promise given on behalf of a public body, or from the existence of a regular practice which a claimant can reasonably expect to continue.” (Lewisham (2013), paragraph 94,).

Although the legal bar for ‘legitimate expectation’ may be higher than that used in non-legal discussions of fairness (Cumming, 2008), the concept is broadly the same. If something about an examination fails to meet those expectations, the examining body will be accused of ‘unfairness’ in the implied contractual sense.

The third sense is relational – treating (relevantly) like cases alike. Almost all assessments involve discrimination of some kind, meaning the identification of distinctions between levels of achievement or between candidates who display different characteristics. In this sense the discrimination is fair if it is based on relevant considerations and unfair if it is based on something else (for example, the candidate’s race or gender). As we shall see this sense lies at the heart of an emerging consensus in assessment theory. It has also featured in the courts – in the UK case referred to above, it was alleged that candidates who sat a particular exam in June 2012 received lower grades than did candidates of a comparable standard who sat the same exam the previous January.

The fourth sense can be described as retributive. An outcome is regarded as fair in this sense if it is an appropriate reward (or penalty) for what has gone before. This sense goes beyond the first, formal, sense and implies that the outcome is deserved and thus, to that extent, justified. In the UK, exam boards and regulators often talk of students getting the ‘grades they deserve’. This can be said in a value-neutral way, simply implying that the requirements for the grade have been met, but more often has evaluative overtones, implying that it is a just reward for the student’s hard work.



 

---

Quality of persuasiveness (25 marks)