August 1, 1998

The Interchangeability of Assessment Methods in Science

Authors:
Brenda Sugrue, Noreen Webb and Jonah Schlackman
This article describes a study that investigated the interchangeability of four different assessment methods for measuring middle-school students’ understanding of science concepts. The four methods compared were hands-on tasks with associated multiple-choice and written justification items, written analogues of the hands-on tasks, and two types of multiple-choice items that were not related to hands-on or written analogue tasks. Some students took the hands-on test before the written test, and some students took the written test before the hands-on test. Observed and disattenuated correlations were examined. Multivariate generalizability analysis was used to obtain disattenuated correlations. The results indicate that hands-on and written analogue tests are not interchangeable, but multiple-choice and written justification items linked to hands-on and written analogues could be considered interchangeable if correlations between .76 and .96 are an acceptable criterion for interchangeability. In addition, a number of interesting order effects were found.
Sugrue, B., Webb, N., & Schlackman, J. (1998). The interchangeability of assessment methods in science (CSE Report 474). Los Angeles: University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
This is a staging environment