Teacher and Leader Effectiveness

Three value-added models – a fixed-effects model, an EVAAS model, and a random effects model – outperformed all others; a final model – a different EVAAS model – was not testable.

Evaluation of nine commonly-used teacher value-added models suggests that, while none of the models performs sufficiently well for high-stakes purposes on their own, four models (two three-level hierarchical linear models, the state’s EVAAS model, and a student fixed effects model) are sufficient for lower-stakes purposes.

Available; posted October 2012

Downloads: Full Report | Executive Summary

The North Carolina Educator Evaluation System was expanded under Race to the Top to include a rating category that formally integrates student growth data. The state also developed new measures of student growth to expand existing measurement tools, as well as a teacher value-added assessment system (EVAAS). Correlations of the student growth rating category with other categories suggest that EVAAS provides an objective measure of teachers’ contributions to student learning. EVAAS measures also are significantly and positively related to a pilot survey that measures student perceptions of their classroom environment, but measures of teaching effectiveness using the CLASS observation tool are not related to EVAAS estimates. Teachers’ perceptions of their own efficacy are significantly and positively related to their value-added estimates. There is limited but promising use of student growth data to inform instruction, though misperceptions and uncertainties related to the new standard as a growth measure raise concerns about the ability to effectively use student growth data to inform instruction.

Downloads: Full Report

North Carolina adopted value-added scores for individual teachers as a sixth standard to supplement its pre-existing five-standard teacher evaluation system. Because almost all identifications of a need for improvement were based on the value-added score, value-added effectively acted alone to determine teachers’ evaluation status, rendering the judgments of principals on other aspects of teachers’ performance less important. Consideration should be given to systematically adding other direct measures of teaching performance in addition to the value-added scores; teachers who do not have scores could then have direct measures of their individual performance incorporated into their evaluations. In addition, it appears that teachers are being rated globally; ratings and feedback are not providing them with actionable information. However, educators appear to feel that the system is fair, and there is no evidence that it has produced negative side effects in terms of teacher behavior. The main issue appears to be that the system includes only one systematic data source, and, while value-added is an important objective measure of effectiveness, the inclusion of additional systematic measures may point out the strengths and weaknesses of individual teachers, increase the accuracy of identifying those who need improvement, increase the favorability of teachers’ attitudes toward the evaluation system, and provide direct information about practices that can be used for improvement for all teachers.

Downloads: Executive Summary

This report examines teachers’ and principals’ perceptions of the inclusion of the new student growth measure (Standard 6) in the evaluation process. Educators used student growth data in three ways: 1) as part of their teacher evaluation discussions (though they wanted more guidance on interpreting Standard 6 scores); 2) as motivation for collaboration among teachers, schools, and districts (though some noted an increase in competitiveness as well); and 3) as a tool for modifying instructional practices. Educator reflections on the use of growth data for evaluation included: 1) a desire for more training on how scores are calculated and on how to use scores to improve instructional practices; 2) appreciation of the additional data to inform instructional practices, but also concern about the impact of an increasingly data-driven school culture; and 3) suggestions for improvement, such as incorporating data that takes student extenuating circumstances into account, including a metric for evaluating teachers-student relationships and interactions, and reducing the weight of the student growth standard in the overall evaluation. Recommendations for the state to consider include: 1) expanding training related to Standard 6; 2) seeking more teacher input on improving the evaluation process; 3) continuing to use feedback loops with teachers; 4) emphasizing the evaluation process as a tool for teacher collaboration; and 5) considering addition of a standard that addresses teacher-student relationships.

Downloads: Full Report

The purpose of this report is to evaluate the effects of adding an eighth standard—school-level Education Value-Added Assessment System (EVAAS) scores—to the evaluation of North Carolina school principals. The report describes the relationship between the principal evaluation ratings and other measures of administrator effectiveness, as well as trends in administrator evaluations between the 2010-11 and 2013-14 school years. Overall, the data and analyses conducted for this report suggest that, in spite of a strong theory that systematic evaluation of principals could lead to improving principals’ performance through the NCSSE ratings, it is unlikely that the system as it is currently implemented will do so. The vast majority of principals receive ratings above Proficient for all standards, even though many schools are classified as performing below expectations. In addition, this evaluation provides some evidence that principals’ rating may not be entirely fair, with principals in schools with higher concentrations of African-American and economically disadvantaged students receive lower ratings. To overcome some of these issues, the report recommends that the state incorporate other measures of principal performance (such as retention of effective teachers, teacher survey ratings of principals’ instructional leadership, and teacher survey ratings of the fairness and feedback provided in teacher evaluations) into a composite quantitative rating of principals’ overall performance.

Downloads: Full Report | Executive Summary

This report offers comparative descriptive analysis between bonus-eligible schools (lowest 5% of elementary, middle, and high schools) and bonus-winning schools (those bonus-eligible schools making “high growth”), as well as qualitative analysis of teachers’ and administrators’ perceptions of and responses to the performance initiative. There is little evidence that performance pay had a causal impact on either teacher effectiveness or student achievement for school years 2010-2011 or 2011-2012: most teachers were not aware their performance was being incentivized; and most teachers reported that a $1,500 bonus would not change their teaching practices. Also, while a few respondents preferred a classroom-based performance structure, the vast majority of teachers and administrators preferred a school-level bonus structure, citing concerns about collaboration and morale. Some recognized the extra pressure teachers in tested subject areas face and were therefore not opposed to those teachers receiving a larger percentage of the bonus money; however, many teachers recognized that it “takes a whole school” to educate a child and believed the bonus initiative should be structured accordingly.

Beginning in 2012, a $500 individual-level incentive was made available to teachers of tested subjects whose classrooms exceed expected growth, regardless of school-wide performance, in addition to the school-wide $1,500 bonus available in designated schools that met high growth. Overall, the performance incentive likely has had limited impact on teacher improvement and student growth. Awareness of the program has increased, but there is still confusion about details, with few educators aware of the availability of the classroom-level bonus. A majority of teachers reported that the incentive did not or would not play any role in their own teaching; however, some teachers—especially at the middle and high school levels—did concede that incentives may impact others’ teaching. When teachers reported significant improvements to either their own or their colleagues’ practice, they often attributed those changes to learning coaches, professional development and training, and collaboration and teamwork—not to the presence of the incentive. If North Carolina intends to continue performance pay for teachers, careful attention to clear, concise, and consistent communication about the program must be a priority. In addition, policy makers may consider directing more resources to supporting the other strategies listed above that teachers indicated were more effective.