Approaches for Combining Multiple Measures of Teacher Performance: Reliability, Validity, and Implications for Evaluation Policy
Educational Evaluation and Policy Analysis
Published online on September 22, 2016
Abstract
A key question facing teacher evaluation systems is how to combine multiple measures of complex constructs into composite indicators of performance. We use data from the Measures of Effective Teaching (MET) study to investigate the measurement properties of composite indicators obtained under various conjunctive, disjunctive (or complementary), and weighted (or compensatory) models. We find that accuracy varies across models and cut-scores and that models with similar accuracy may yield different teacher classifications. Accuracy and consistency are greatest if composites are constructed to maximize reliability and lowest if they seek to optimally predict student test scores. We discuss the implications of the results for the validity of inferences about the performance of individual teachers, and more generally for the design of teacher evaluation systems.