June 27, 2019
Ensuring the Comparability of Modified Tests Administered to Special Populations
Authors:
Phoebe C. Winter, Mark Hansen, and Michelle McCoy
In order to accurately assess the English language proficiency of special populations of English learners, student assessment programs must maintain the comparability of standard and modified assessment formats, allowing for equivalent inferences to be made across student classifications. However, given the typically small size of special populations of English learners, such as blind and low vision students, traditional calibration and item linking techniques are often incapable of ensuring sufficient levels of comparability. With this in mind, researchers at CRESST set out to test the efficacy of a new item calibration technique: one in which the overall cut scores on the ELPA21 braille form require the same language skills and knowledge as the cut scores on the ELPA21 online form.
To test their approach, the researchers recruited a panel of six educators who specialize in working with special populations of students. Using data from the 2016-2017 braille test administration, the panelists reviewed the similarities and expected differences in difficulty between test form items. They went on to estimate the proportion of students taking the braille form who would meet the target student descriptors and correctly responded to test items. In doing so, the panel was able to successfully recommend cut scores for the braille form while preserving the necessary comparability with cut scores on the online form. This process could be utilized for other assessments in which modified test items are present and cannot be said to have the same parameters as their source items.
Winter, P. C., Hansen, M., & McCoy, M. (2019). Ensuring the comparability of modified tests administered to special populations (CRESST Report 864). Los Angeles: University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).