September 1, 2013

Treatment Confounded Missingness: A Comparison of Methods for Addressing Censored or Truncated Data in School Reform Evaluations

Authors:
Jordan H. Rickles, Mark Hansen and Jia Wang
In this paper we examine ways to conceptualize and address potential bias that can arise when the mechanism for missing outcome data is at least partially associated with treatment assignment, an issue we refer to as treatment confounded missingness (TCM). In discussing TCM, we bring together concepts from the methodological literature on missing data, mediation, and principal stratification. We use a pair of simulation studies to demonstrate the main biasing properties of TCM and test different analytic approaches for estimating treatment effects given this missing data problem. We also demonstrate TCM and the different analytic approaches with empirical data from a study of a traditional high school that was converted to a charter school. The empirical illustration highlights the need to investigate possible TCM bias in high school intervention evaluations, where there is often an interest in studying the effects of an intervention or reform on both school persistence and academic achievement.
Rickles, J. H., Hansen, M., & Wang, J. (2013). Treatment confounded missingness: A comparison of methods for addressing censored or truncated data in school reform evaluations (CRESST Report 832). Los Angeles: University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
This is a staging environment