Overview | Evaluation Reports
The evaluation of the Race to the Top District and School Transformation (DST) initiative is two-fold. First, the evaluation will seek to understand the theory of action that governs DST work and the degree to which aspects of the DST unit’s approaches have led to desired school and LEA improvements. Second, the evaluation will explore the extent to which and the ways in which RttT-funded interventions by the District and School Transformation division (DST) improve outcomes for students in the state’s lowest-performing schools and districts. The approach will combine qualitative explorations (such as school and LEA site visits and interviews) with quantitative analyses (including analyses of educator and student survey results, and assessments of changes over time in student outcomes). The evaluation also will provide formative feedback to the DST unit to inform future development of its approaches to intervention.
RttT Initiative Context Policy Objective(s)/Purpose(s) of the Initiative
- Dramatically improve achievement in the lowest 5% of schools in NC where Performance Composites are below 50% proficient and improve graduation rates in high schools where the graduation rate is under 60%;
- Raise district-wide performance in those districts with a high concentration of lowest-achieving schools; and
- Through these steps, provide new opportunities for students in the lowest-achieving schools and districts to attend schools that will better support their achievement and successful graduation and lead them to college and career readiness.
Evaluation Reports | Overview
|Evaluation of District and School Transformation RttT Work|
|1. Turning Around North Carolina’s Lowest Achieving Schools (2006-2010)
There is evidence of success in providing high-quality assistance to most targeted low-achieving schools. There is also evidence of several undermining factors in the schools that made little or no progress.
|Available; posted September 2011|
|2. Productive Connections: Interventions in Low Performing Districts by NCDPI District and School Transformation Division in 2011-12
This second report focuses on the District and School Transformation (DST) division’s district-level work in the twelve lowest-achieving school districts in the state. It finds that, in low-achieving districts, connections are weak or missing between and within central offices, schools, and classrooms. DST’s strategy at the district level is to strengthen or create productive connections across and between levels of the systems, thus complementing its strategy of scaffolded craftsmanship for supporting change in low-achieving schools in those districts. In sum, “district transformation” is essentially the process of changing a disconnected district into a productively connected district. Early qualitative and quantitative evidence suggests that DST is making a measurable contribution to the improvement of student performance rates and student graduation rates in the schools it serves.
|Available; posted April 2013|
|2a. Turning around North Carolina’s Lowest-Achieving Schools: Initial Findings on the School Leader Professional Development Series
This report presents findings from DST’s School Leaders Professional Development Series. Data suggest that, overall, DST successfully constructed and delivered a high-quality professional development program to participants through the series. To strengthen the series, this report recommends that DST: (1) provide differentiated professional development offerings on the basis of participants’ levels of leadership experience as well as each school’s level of student achievement progress; and (2) provide literacy-focused professional development with high-quality facilitators who have recent and relevant experience in sound, research-based practices and with materials differentiated by school
|Available; posted April 2013|
|3. Evaluation of District and School Transformation School-Level Coaching and Professional Development Activities
New assessments in 2012-13 contributed to declines in Performance Composite scores for all schools statewide, but schools targeted by DST mostly experienced smaller declines. By spring 2013, DST schools registered higher levels on two of 19 dimensions of leadership and organization—teacher knowledge-sharing and use of formative assessment—than did comparison schools. However, DST target schools improved at a slower rate on two other measures: teacher-leader respect and team orientation. Principals of DST schools agreed that their DST coaches helped them do a better job, and they rated their coaching higher than did comparison principals in terms of improving shared leadership and order and suggesting actionable approaches or solutions to challenges. However, they reported no differences in coaching impact on improving teaching and assessment practices or improving teacher efficacy and responsibility. Less-experienced and lower-skilled DST principals’ ratings of their coaches’ effectiveness were higher than the ratings given by more experienced, higher-skilled principals. Teachers’ ratings of teacher-leader respect and team orientation declined in DST schools, though changes cannot be attributed to any one cause. Overall, DST professional development sessions were highly rated by participants, and the eight characteristics of quality professional development were present in the majority of observed segments.
|Available; posted October 2014|
|4. Outcomes and Impacts of North Carolina’s Initiative to Turn Around the Lowest-Achieving Schools
North Carolina’s lowest-achieving schools in 2009-10 improved their performance during the four years of TALAS. In addition, DST efforts to emphasize literacy have paid dividends in increased proficiency in that area in elementary and middle schools when compared to other low-performing schools, and not at the cost of other areas (science and mathematics). Many TALAS high schools made large gains in their graduation rates, though the effects of TALAS are difficult to distinguish from the nearly ten-percentage-point increase in the statewide graduation rate during RttT. It appears that the effects of TALAS are larger when district-level coaching and support are included with school leadership and instructional coaching, and when schools participated in both the earlier transformation program and TALAS, suggesting that sustained services contribute to greater growth. Another contributing factor to the success of TALAS appears to be the Comprehensive Needs Assessments and School Improvement Plans that were developed in the first year of TALAS—a component not present in pre-RttT transformation efforts. There is some evidence that, in order to sustain the positive effects of TALAS, the state may need to continue to intervene to transform low-performing schools for the foreseeable future.
|Available; posted September 2015|
|Evaluation of STEM Anchor School System Development|
|1. North Carolina’s STEM High Schools: An Overview of Current Data
STEM schools serve greater proportions of lower-income, minority, and rural students. Student performance is higher in high-minority STEM (vs high-minority non-STEM) schools.
|Available; posted December 2011|
|2. STEM Affinity Networks: Year 1 Report
Most Year 1 implementation activities were high-quality; however, delays in selecting schools for the network and in agreeing on curriculum development expectations hindered optimal implementation.
|Available; posted April 2012|
|3. STEM Affinity Network: Year 2 Report
Anchor schools have improved instruction and implemented STEM features (such as project-based learning and partnering for improvement of student learning) internally. Structures for networking, PD, curriculum development, and partnerships are in place to support affinity schools, though some of these activities have been delayed. There is not yet universal buy-in among anchor school staff, but many report improved instruction and implementation of STEM strategies. Students in anchor schools receive personalized attention and exhibit high motivation, engagement, and a passion for learning.
|Available; posted November 2012|
|4. STEM Affinity Network: Year 3 Report
Structures for networking, professional development, curriculum development, and partnerships are in place to support STEM anchor and affinity schools. Some implementation areas progressed more than others, and some components of each area also developed faster than other components. Staff and student surveys reveal that after one year of implementation, a subset of the affinity schools (the comprehensive schools) lags behind the anchor schools and the other affinity schools (small new schools and STEM Academies) in all four areas of implementation, as well as in intended student outcomes. Remaining implementation challenges include: 1) logistical, time, and resource challenges; 2) student, faculty, and community buy-in; 3) implementation of STEM curriculum and instruction; 4) sustainability; and 5) relationships with the wider community.
|Available; posted December 2013|
|5. STEM Affinity Network: Final Report
Between 2010-11 and 2012-13, staff and students reported a more STEM-focused environment, and teachers’ reported use of targeted instructional practices was higher. In most cases, the levels of implementation of short-term outcomes reported by teachers and students were higher in anchor schools, small new schools, and STEM academies, but the greatest positive changes were in comprehensive high schools. In the anchor schools, staff reported that they need more time and support in order to become model STEM schools; however, there were notable improvements in instruction in anchor school classrooms, and all four anchor schools have developed strong partnerships with industry and institutions of higher education. The frequency of networking among staff remained unchanged between 2011-12 and 2012-13 and happened mostly during professional development events. North Carolina New Schools met or exceeded its obligations for coaching and professional development, and the majority of school staff agreed that these services were good. Sixteen integrated STEM courses were completed by the North Carolina School of Science and Mathematics. Participation of school staff in collaborations with business and community partners significantly increased in all types of schools. Overall, RttT STEM funds supported the creation of a state-wide STEM strategic plan.
|Available; posted December 2014|