jeromedelisle.org

There has been a shift away from using the term item bias to Differential Item Functioning. DIF is distinct from impact and considered a threat to validity. Item bias describes a situation where the source of DIF is some characteristic of the test item or testing situation not relevant to the ability of the candidate. DIF is a necessary but insufficient condition for item bias. Item impact is DIF when there are true differences between the underlying ability of interest. This might occur because of differences in training and learning.

Review 1

1. An item is considered to show differential functioning (DIF) if the item response function is the same across groups (Zwick, 1990).

2. DIF means that either the item performs differently or measures something different. If the item shows DIF it means that the item is less valid for one subgroup (Steinberg & Thissen, 2006).

3. A fundamental aspect of all definitions of DIF is the matching of students in the reference and focal groups on some measure of ability (Clauser & Mazor, 1998).

4. DIF is different to differential test functioning (DTF), which focuses upon the collection of test items (Camilli & Penfield, 1997).



Three generations of DIF research are identified by Zumbo (2007).

1. Concept Development.

2. Statistical Methods

3. Explanations for DIF and integration into practice and use of testing.

Concept Development

Concerns for bias increased in the 1960s during the US civil rights era- issues were related to the differential performance on high stakes test related to sex and race. Defined as predictive validity, the focus on bias increased as IQ tests were used for selection and placement.  Were differences really related to bias?

There was a turn away from use of the term bias in technical decisions to Differential Item Functioning.This might have been in part related to the highly politicized environment.

DIF means that a candidate from a group answers a question corrently more often than an equally knowledgeable person from another group. DIF is the circumstance in which two individuals of similar ability do not have the same probability of answering a question in a particular way. This often is examined to assess whether men and women or individuals of different ethnicity are likely to provide disparate answers on a test. If so, the fairness of the test can be called into question.

Uniform DIF occurs when there is no interaction between ability level and group membership. Nonuniform DIF occurs when there is interaction between ability level and group membership.

Statistical Methods

Item bias were first treated as item by group interaction. This was followed by chi-square and IRT approaches. Lord defined DIF as a difference in ICCs for the item. By the 1990s there were multiple methods-Mantel-Haenszel procedure, logistic regression, standardization, Lord's chi-square, Raju's area method, and the likelihood ratio test. The logistic regression procedure is the most flexible and the MH the most widely used.

Explanations for DIF and integration into practice and use of testing.

At the end of the 1990s, attempts at explaining DIF had met mostly with failure. There was the concept of undesired multidimensionality. More comprehenive ecological models became in vogue. This era saw the introduction of structural equation modeling as a tool to explain multiple causes.

Review 2

•The question of why a particular item displays gender DIF is not always answerable even from the international studies.  Therefore, the interpretation and explanation of the statistically results is the most difficult part of a gender DIF study (Takala & Kaftandjieva, 2007).

•Pae (2007) examined the examined potential sources of gender DIF on a high-stakes national language test. Current theory suggests that possible influences include (1) the level and content of items, (2) the skills evoked, (3) familiarity and interest with the content, and (4) examinees perceptions and understanding of test items.





Example from National Learning Assessments TTO

Item Bias & Sensitivity Reviews

Item Bias can be dealt with in the early stages of test development. A bias and sensitivity review helps to ensure that items and stimuli are free of bias and do not exclude any group of children from demonstrating what they know and are able to do. As part of the Sensitivity review, items are checked for a variety of cultural, regional, philosophical, political, and religious backgrounds throughout Florida.

Gender
The following should be considered when reviewing items:
• What terms are used to refer to humanity at large?
• In what activities are boys and girls involved?
• What emotions do characters display?
• What situations are characters placed in?
• How are pictures or visuals used?

Race, Ethnicity, or Culture
The following should be considered when reviewing items:
• Portrayal of any group or group member in a demeaning, offensive, condescending, or insensitive way
• Stereotyping any groups with respect to activities, emotions, language, or characteristics
• Use of pictures that do not represent the diversity of the student population
• Over‐inclusion or under‐inclusion of any group

 Some questions to ask during the review:
• How are different ethnic groups or members of ethnic groups portrayed?
• Is there any stereotyping with respect to activities, emotions, or characteristics?
• How varied are the pictures used to represent the diversity of the student population?
• Is any group over‐included or under‐included?

Economic or Social Class
The following should be considered when reviewing items:
• Contains an activity familiar to only one class or an activity that is not familiar to all classes
• Portrayal of leisure activities that aren’t accessible to all children
• Preference or dominance of values from any one class

Smarter Balanced Bias and Sensitivity Reviews

 Developing an Item Boias Review Form