September 24, 2025

Using Learner-System Interactions as Evidence of Student Learning and Performance: Validity Issues, Examples, and Challenges

rainbow divider
Authors:
Gregory K. W. K. Chung, Tianying Feng, and Elizabeth J. K. H. Redman
This chapter explores the idea of using learner-system interactions as a source of evidence about students’ learning and performance in the context of Principle 3: Assessment design supports learners’ processes, such as motivation, attention, engagement, effort, and metacognition; and Principle 4: Assessments model the structure of expectations and desired learning over time. We illustrate how well-designed instructional opportunities in interactive digital environments naturally provide measurement opportunities. These opportunities can result in what we call “measurement without testing”: Learner-system interactions that are designed to support students’ learning are by definition observable and we believe carry the most relevant information about students’ learning. Digital environments enable the collection of finegrained behavioral data about what, when, and how a learner interacts within that environment.

However, for learner-system interactions to serve as evidence, three design challenges must be addressed: identifying the cognitive demands of the task, identifying the learning-relevant indicators of interest, and developing algorithms to transform low-level behavioral events into high-level indicators that represent learning-relevant processes. If we can observe what learners are doing as they do it and develop the methodology to accurately determine why, then that capability may help move us toward tailored, adaptive, and individualized learning for all students.
Chung, G. K. W. K., Feng, T., & Redman, E. J. K. H. (2025). Using learnersystem interactions as evidence of student learning and performance: Validity issues, examples, and challenges. In S. G. Sireci, E. M. Tucker, & E. W. Gordon (Eds.), Handbook for assessment in the service of learning, Volume II: Reconceptualizing assessment to improve learning. University of Massachusetts Amherst Libraries.
rainbow divider
This is a staging environment