jeromedelisle.org

School of Education
University of the West Indies, St. Augustine
St. Augustine

ph: 868-477-1500

delislejerome@gmail.com

  • Home
  • My Life in Pictures
  • The Research PageClick to open the The Research Page menu
    • List of Publications
  • EDEA 6202-Educational Improvement Project 2020Click to open the EDEA 6202-Educational Improvement Project 2020 menu
    • School Self Evaluation
    • Step by Step
  • EDRS 6910-Mixed Methods Research Designs: Issues and ApplicationsClick to open the EDRS 6910-Mixed Methods Research Designs: Issues and Applications menu
    • MMR-AN INTRODUCTION
    • Course Calendar
    • MMR-History & Controversies
    • Readings
    • Designs
    • Assignment
    • Paradigms/ Paradigmatic Frameworks
    • Integration in MMR
  • EDME 6121-Evaluation of Educational InstitutionsClick to open the EDME 6121-Evaluation of Educational Institutions menu
    • EDME 6121-Evaulation of Educational Institutions (Introduction)
    • EDME 6121-COURSE ASSESSMENT & EXAMINATIONS
    • EDME 6121-GENERAL TOPICS IN EVALUATION
    • EDME 6121-National Systems of Evaluation in Education
    • EDME 6121-International Benchmarking Page
    • EDME 6121-Programme Evaluation Page
    • EDME 6121-Logic Modelling
    • EDME 6121- Models & Designs
    • A Trinidad and Tobago Evaluation Association?
  • EDEA 6201-SCHOOL AND SYSTEM IMPROVEMENTClick to open the EDEA 6201-SCHOOL AND SYSTEM IMPROVEMENT menu
    • Course Content
    • Keep up-to-date
    • Theory Page
    • Key Readings
    • History Videos
    • Events
    • High Performing School Systems-Read & Watch
    • Evidenced-informed system and school improvement
    • The coherence framework
    • How School Systems Improve
    • Benchmarking for system improvement
    • Educational Improvement-Improving Learning Ecosystems
    • High Performing Systems-More Videos
    • Even More Videos-Snapshots of Improvement
    • Strong Performers and Successful Reformers in Education (OECD PISA)
    • Innovative school change
    • School Turnaround theory
    • Improving Teacher Preparation Systems
    • Assignment Page
  • EDME 6300-Measurement TheoryClick to open the EDME 6300-Measurement Theory menu
    • Course Schedule
    • Measurement Publications
    • People in Measurement
    • Validity Theory
    • Validity Theory Applied to the Caribbean
    • The Validity Debates
    • The Validation Process
    • Test Fairness
    • Item and test bias
    • Assignment Page
  • EDEA 6200-ACTION RESEARCH FOR SCHOOL IMPROVEMENTClick to open the EDEA 6200-ACTION RESEARCH FOR SCHOOL IMPROVEMENT menu
    • What is Action Research?
    • History & Thinkers
    • Books & References
    • The Reading Room
    • Nuts & Bolts
    • Whole School Interventions
    • SAMPLE STUDY
    • The Viewing Room
    • Brainstorming Strategies
    • Assignment Page
    • Assignment Rubric Page
    • Giving Presentations
    • Starting up a PLC
    • What Works Page-
    • Improvement & Implementation Science Tools
    • Research & Evidence in the Schools
    • The Class of 2016 Issues
  • EDME 6006-ASSESSMENT & EVALUATION-2016Click to open the EDME 6006-ASSESSMENT & EVALUATION-2016 menu
    • Updates-week by week
    • TOBAGO SCHEDULE
    • EDME 6006-ASSIGNMENTS
    • A Model for Classroom Assessment In Trinidad & Tobago
    • Assessment System-Purposes and Types
    • Issues in assessment
    • New videos on classroom assessment
    • More videos on classroom assessment
    • Experts on Formative Assessment
    • Resources & Support-Formative Assessment & Item Writing
    • Resources & Support-A call for extended performance assessments
    • Resources and Support-All about rubrics
    • Assessment in the New Primary School Curriculum
    • Item Analysis Support
    • Test Fairness & Validity
    • More on test validity
    • NEW PRACTICE QUESTIONS
    • Resources & Support-Measurement Topics
  • Friends and Family

Item and test bias

There has been a shift away from using the term item bias to Differential Item Functioning. DIF is distinct from impact and considered a threat to validity. Item bias describes a situation where the source of DIF is some characteristic of the test item or testing situation not relevant to the ability of the candidate. DIF is a necessary but insufficient condition for item bias. Item impact is DIF when there are true differences between the underlying ability of interest. This might occur because of differences in training and learning.

 

 


Review 1

  1. An item is considered to show differential functioning (DIF) if the item response function is the same across groups (Zwick, 1990).

  2. DIF means that either the item performs differently or measures something different. If the item shows DIF it means that the item is less valid for one subgroup (Steinberg & Thissen, 2006).

  3. A fundamental aspect of all definitions of DIF is the matching of students in the reference and focal groups on some measure of ability (Clauser & Mazor, 1998).

  4. DIF is different to differential test functioning (DTF), which focuses upon the collection of test items (Camilli & Penfield, 1997).




Three generations of DIF research are identified by Zumbo (2007).

1. Concept Development.

2. Statistical Methods

3. Explanations for DIF and integration into practice and use of testing.

 

Concept Development

Concerns for bias increased in the 1960s during the US civil rights era- issues were related to the differential performance on high stakes test related to sex and race. Defined as predictive validity, the focus on bias increased as IQ tests were used for selection and placement.  Were differences really related to bias?

There was a turn away from use of the term bias in technical decisions to Differential Item Functioning.This might have been in part related to the highly politicized environment.

DIF means that a candidate from a group answers a question corrently more often than an equally knowledgeable person from another group. DIF is the circumstance in which two individuals of similar ability do not have the same probability of answering a question in a particular way. This often is examined to assess whether men and women or individuals of different ethnicity are likely to provide disparate answers on a test. If so, the fairness of the test can be called into question.

 

Uniform DIF occurs when there is no interaction between ability level and group membership. Nonuniform DIF occurs when there is interaction between ability level and group membership.

 

 

Statistical Methods

Item bias were first treated as item by group interaction. This was followed by chi-square and IRT approaches. Lord defined DIF as a difference in ICCs for the item. By the 1990s there were multiple methods-Mantel-Haenszel procedure, logistic regression, standardization, Lord's chi-square, Raju's area method, and the likelihood ratio test. The logistic regression procedure is the most flexible and the MH the most widely used.

 

Explanations for DIF and integration into practice and use of testing.

At the end of the 1990s, attempts at explaining DIF had met mostly with failure. There was the concept of undesired multidimensionality. More comprehenive ecological models became in vogue. This era saw the introduction of structural equation modeling as a tool to explain multiple causes.


Review 2

•The question of why a particular item displays gender DIF is not always answerable even from the international studies.  Therefore, the interpretation and explanation of the statistically results is the most difficult part of a gender DIF study (Takala & Kaftandjieva, 2007).

•Pae (2007) examined the examined potential sources of gender DIF on a high-stakes national language test. Current theory suggests that possible influences include (1) the level and content of items, (2) the skills evoked, (3) familiarity and interest with the content, and (4) examinees perceptions and understanding of test items.

•The specific content and cognitive skills have also been shown to produce gender DIF in mathematics items. The items’ cognitive complexity may be a factor based upon the extent to which computation, comprehension and analysis is evoked. 

•For language tests, Gender DIF may be generated on some specific questions or because of the examinee’s familiarity, interest and negative emotions. DIF might also be generated from the frequencies of local and global reading strategies.





 



Example from National Learning Assessments TTO

 

 

Item Bias & Sensitivity Reviews

Item Bias can be dealt with in the early stages of test development. A bias and sensitivity review helps to ensure that items and stimuli are free of bias and do not exclude any group of children from demonstrating what they know and are able to do. As part of the Sensitivity review, items are checked for a variety of cultural, regional, philosophical, political, and religious backgrounds throughout Florida.

 

Gender
The following should be considered when reviewing items:
• What terms are used to refer to humanity at large?
• In what activities are boys and girls involved?
• What emotions do characters display?
• What situations are characters placed in?
• How are pictures or visuals used?


Race, Ethnicity, or Culture
The following should be considered when reviewing items:
• Portrayal of any group or group member in a demeaning, offensive, condescending, or insensitive way
• Stereotyping any groups with respect to activities, emotions, language, or characteristics
• Use of pictures that do not represent the diversity of the student population
• Over‐inclusion or under‐inclusion of any group

 Some questions to ask during the review:
• How are different ethnic groups or members of ethnic groups portrayed?
• Is there any stereotyping with respect to activities, emotions, or characteristics?
• How varied are the pictures used to represent the diversity of the student population?
• Is any group over‐included or under‐included?


Economic or Social Class
The following should be considered when reviewing items:
• Contains an activity familiar to only one class or an activity that is not familiar to all classes
• Portrayal of leisure activities that aren’t accessible to all children
• Preference or dominance of values from any one class

 

Smarter Balanced Bias and Sensitivity Reviews

 Developing an Item Boias Review Form

 

Copyright 2009 jeromedelisle.org. All rights reserved.

Web Hosting by Yahoo!

School of Education
University of the West Indies, St. Augustine
St. Augustine

ph: 868-477-1500

delislejerome@gmail.com