July 2, 2002
Conceptual Framework and Design of the High School Study: A Multidimensional Approach to Achievement Validation
Authors:
Richard J. Shavelson, Robert Roeser, Haggai Kupermintz, Shun Lau, Carlos Ayala, Angela Haydel, and Susan Schultz
Richard E. Snow conceived of an individual’s performance as the result of a transaction between her aptitudes and the particular characteristics of the situation in which that performance occurs over time. By aptitudes, he meant the cognitive and motivational resources that an individual brings to the situation. By situation, he meant the characteristics of a particular environment that “afforded or impeded”—that assisted or constrained—certain courses of goal-related action.
When Snow applied his ideas to achievement testing situations, he recognized that test performance resulted from a student’s background and intellectual history, as well as the cognitive and motivational resources that the student cobbled together to respond to a series of situation-embedded test tasks (e.g., multiple-choice items or performance assessments). In essence, Snow’s idea was that individuals’ achievement test performance depended not just on their knowledge and abilities, but rather on a full spectrum of interrelated, situation-relevant cognitive and motivational resources that transacted with the affordance, constraint, and demand structures of the testing tasks themselves.
From this reasoning and from empirical findings, he concluded that a new multivariate approach to validating interpretations of achievement test scores was needed. We first set forth in more detail Snow’s new aptitude theory as it applies to multivariate test validity and then describe the design of the “High School Study” we conducted as a step in exploring his ideas about validity.
Shavelson, R. J., Roeser, R., Kupermintz, H., Lau, S., Ayala, C., Haydel, A., & Schultz, S. (2002). Conceptual framework and design of the high school study: A multidimensional approach to achievement validation (CSE Report 569). Los Angeles: University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).|Shavelson, R. J., Roeser, R., Kupermintz, H., Lau, S., Ayala, C., Haydel, A., & Schultz, S. (2002). Conceptual framework and design of the high school study: A multidimensional approach to achievement validation (CSE Report 569). Los Angeles: University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).