jeromedelisle.org

Course Rationale
The area of educational measurement is critical to the applied areas of assessment, evaluation, and psychology. Students in these areas must understand the modern principles of measurement and be able to apply different measurement constructs to the development, choice, and evaluation of data collection instruments and exercises.

Capacity in educational measurement requires the development of several practical competencies, including use of statistical software, gained through experiential learning. Although a mathematical or statistical background will prove helpful, students who show aptitude and a keen interest in the field will not be excluded from the field.

Course Description
Measurement means assigning numbers in an attempt to differentiate the values for some targeted variable. Measurement theory in education refers to the set of principles and procedures that governs the procedures for using educational tests and assessments. According to Snow and Lowman (1989), this field has advanced significantly in the last two decades and now includes a body of both declarative and procedural knowledge.

This programme provides extensive training on the basic principles and core constructs in modern measurement theory, with an analysis of themes on validity and reliability, the process of test as validation, test fairness, standard setting, item response theory, and equating.

The programme also emphasizes training in the use of software for test equating, item analysis using item response theory, and differential item functioning. The programme is taught through multiple approaches, including journal readings, discussions, lectures and laboratory sessions. Assessments are extended and performance-based, focused on practical problems experienced in the real world of educational measurement.

Aim/Goal

To develop a conceptualization of basic measurement principles and practical
competencies, including use of software applicable to measurement.

Learning Outcomes

At the end of the course, the participants will be able to:

1. solve common measurement issues in testing and evaluation;
2. conduct validation of commercial and achievement tests; and
3. use software to analyze items and tests.

Course Content
1) The nature of measurement

a. Test Scores
b. Statistics
c. Scales of Measurement
d. Classical and New Test Theory

2) Validity & Validation

a. Changing Perspectives on Validity
b. Validation as a Process
c. Argument based Approaches to Validation
d. Emerging Issues in Test Use/Interpretation

3) Fairness & Test Bias

a. Understanding fairness
b. Measuring Differences in Test Scores
c. Measuring Differential Performance of Items
d. Using DIF software

4) Reliability and Generalizability

i. Theory
ii. Comparison
iii. Software

5) Norming, Standard Setting and Grade Boundary Setting

a. Methods
b. Protocols
c. Validating

6) Linking, Scaling, & Equating Test Scores

a. Methods
b. Software

7) Classical Item Analysis and Item Response Theory

a. Classical Item Analysis
i. Theory and indices
ii. Software
b. IRT
i. Theory
ii. Models applicable to MCs
iii. Polytomous and mixed models
iv. IRT Software
1. BILOG, IRT Pro, MULTILOG
2. XCALIBRE

Four Definitions of Measurement (Educational Measurement for Applied Researchers)

1.  Measurement is a procedure for the assignment of numbers to specified properties of experimental units in such a way as to characterise and preserve specified relationships in the behavioural domain.Lord, F., & Novick, M. ( 1968 ) Statistical Theory of Mental Test Scores, p.17.

2.  Measurement is the assigning of numbers to individuals in a systematic way as a means of representing properties of the individuals. Allen, M.J. and Yen, W. M. (1979 ). Introduction to Measurement Theory, p 2.

3.  Measurement consists of rules for assigning numbers to objects in such a way as to represent quantities of attributes. Nunnally, J.C. & Bernstein, I.H. ( 1994 ) Psychometric Theory, p 1.

4.  Measurement begins with the idea of a variable or line along which objects can be positioned, and the intention to mark off this line in equal units so that distances between points on the line can be compared.Wright, B. D. & Masters, G. N. (1982) Rating Scale Analysis, p. 1