MetaTOC stay on top of your field, easily

Finding Efficiency in the Design of Large Multisite Evaluations: Estimating Variances for Science Achievement Studies

American Journal of Evaluation

Published online on

Abstract

Little is known empirically about intraclass correlations (ICCs) for multisite cluster randomized trial (MSCRT) designs, particularly in science education. In this study, ICCs suitable for science achievement studies using a three-level (students in schools in districts) MSCRT design that block on district are estimated and examined. Estimates of ICCs, by district, are computed for unconditional two-level (students in schools) hierarchical linear models using student-level science achievement raw scores as measured by the Texas Assessment of Knowledge and Skills in 2010–2011 for Grades 5, 8, 10, and 11. The average within-district ICCs for MSCRTs of science achievement range from 0.0781 to 0.0982 across grades. For Grade 5, a significant difference is found in the ICC estimate suitable for MSCRTs with many districts versus only a few districts. Consequently, evaluators of science education interventions should exhibit care when selecting ICC estimates for three-level MSCRT designs that block on district.