Overview | Evaluation Reports
Formative Evaluation Focus: The successful transition to new standards and assessments and the use of data to improve instruction are ultimately dependent upon the effectiveness of the professional development, technical assistance, and support materials provided to educators across the state. Thus, the focus of this formative evaluation is to provide feedback to state leaders on the quality of professional development activities and resources provided at the state level in support of major RttT initiatives.
Summative Evaluation Focus: Over the next three years, a large portion of NC’s RttT funds will support professional development efforts for teachers, principals, and other education leaders across the state. Data will be collected across initiatives about quality and participation in local and state supported professional development activities, including standards and assessments, instructional data use, NC Virtual Public School, and Turning Around the Lowest Achieving Schools and NC STEM Schools Network. In addition information will be collected through district and school field visits and the cross-cutting teacher and principal surveys to inform summative evaluation analyses examining the extent to which participation in high-quality professional development activities provided through RttT funds have increased the capacity of the education workforce to deliver effective instruction, and, ultimately, to increase in student performance.
RttT Initiative Context
Policy Objective(s)/Purpose(s) of the Initiative
Formative Evaluation Focus:
- Transitions to New Standards and Assessments: North Carolina will transition to the Common Core Standards and other new State standards, and high-quality assessments tied to these standards, by working in partnership with LEAs.
- Using Data to Improve Instruction: Every teacher and instructional leader in NC will have ready access to a high-quality instructional improvement system containing assessment and data analysis tools and guidance in how to use these tools to improve instructional practices.
- Providing Effective Support to Teachers and Principals: Through the Professional Development Initiative (PDI), educators will have access to a comprehensive, targeted, seamless, and flexible professional development system for all educators. The PDI also will develop multiple resources (e.g., Professional Development Leaders, eLearning tools, etc.) in support of these efforts.
Summative Evaluation Focus:
- Provide effective teachers via virtual and blended courses: The NC Virtual Public School (NCVPS) provides courses that augment those available locally to equalize educational opportunities statewide and, in many cases, provide an effective online teacher when a qualified teacher is not available locally. The NCVPS is committed to raising achievement and closing learning gaps with 21st-century innovation by providing access to world-class learning opportunities for all NC students.
- Turning Around the Lowest-Achieving Schools in NC: NCDPI’s District and School Transformation division, along with other change partners, has been engaged in turnaround efforts that closely resemble the four RttT models. As part of the State’s RttT plan, District and School Transformation will transition to implementing the four school intervention models outlined by the USED. Professional development through coaches and other approaches is an important component of the District and School Transformation efforts.
- NC STEM School Network: Work with partners to support the development of a small set of anchor/model STEM high schools that will serve as laboratory schools and sites for professional development around project-based learning.
- Local RttT funding for professional development initiatives: Many school districts have chosen to allocate a portion of the funds allocated to them through RttT to professional development activities.
Evaluation Reports | Overview
|Report on feasibility of using high-tech recording systems for recording teachers’ practice
Download: Full Report
|Available; posted October 2011|
|1. Annual Report: Year 1|
| 1A. Building LEA and Regional Professional Development Capacity: First Annual Evaluation Report
PD ratings were positive, especially among Region 8 (Far West) and elementary teachers. Principals reported that the new Teacher Evaluation Process informed PD planning, but less than 50% reported that staff had received PD on RttT focus areas (standards, assessment, data use). Results were consistent with Teacher Working Conditions survey results.
|Available; posted January 2012|
|1B. Distinguished Leadership in Practice (DLP): First Annual RttT Evaluation Report
In Year 1, DLP sessions were conducted for 194 principals, with 157 completing the program. Principal feedback and observation data indicate that face-to-face and online sessions were of high quality. Principal feedback and focus group data suggest that participants developed specific leadership knowledge and skills
|Available; posted September 2012|
| 1C. Race to the Top Online Professional Development Evaluation: Year 1 Report
Approximately half of the state’s educators completed at least one online module by the end of the 2011-12 school year. Overall, most participants agreed that the webinars and modules were relevant to their professional development needs, though some content was redundant with prior PD activities and not always content- or grade-specific, and some modules did not meet national standards. Most online modules were completed independently and not in Professional Learning Community groups.
|Available; posted December 2012|
|2. Annual Report: Year 2|
|2A. Second Annual Race to the Top Evaluation Report Part I—Statewide Face-to-Face Professional Development Formative Evaluation
This report documents the current status various components of the State’s Annual Professional Development Cycle (APDC). Data indicate that the overall quality of 2011-12 APDC events was high, with some variation across session type. In addition, the State is clarifying state and local professional development needs, though providing adequate time and resources to meet these needs remains a challenge. State-provided professional development in 2011-12 focused on four Race to the Top priorities: (1) transition to new state standards; (2) implementation of formative and summative assessments; (3) use of data to support instruction; and (4) effective use of the North Carolina Educator Evaluation System. Participants generally believed that these events helped them develop knowledge and skills related to these priorities.
|Available; posted March 2013|
|2B. Second Annual Race to the Top Evaluation Report Part II—Local Outcomes Baseline Study
This report provides a baseline assessment of the State’s progress toward updating the education workforce in North Carolina via an in-depth analysis of data from a representative sample of 27 schools. The report examines: increases in leadership capacity to provide high-quality professional development at the local level; changes in school and organizational cultures; and improvements in teachers’ classroom practices. Overall, schools and districts appear to have taken initial steps toward crafting processes for providing professional development that supports Race to the Top priorities (transition to new state standards, implementation of formative and summative assessments, use of data to support instruction, and effective use of the North Carolina Educator Evaluation System).
|Available; posted March 2013|
|3. Annual Report: Year 3|
|3A. Distinguished Leadership in Practice (DLP): Second Annual Evaluation Report
DLP has designed and implemented a high-quality program that aligns to national professional development standards and meets the professional development needs of the participating school leaders. Participants reported that they are building intended knowledge and skills and are improving the culture in their schools. The 2012-13 program served 167 principals, 135 of whom completed all six components. Most participants (80%-99%) agreed or strongly agreed (and observational data supported) that DLP sessions were of high quality, were relevant to their professional development needs, provided them with useful resources, and were led by effective facilitators. Nearly half (47%) of those who had room for improvement increased their leadership level over the course of their year in DLP; however, changes in participants’ demonstrated leadership over the course of their year in DLP were similar to changes evident for non-participating principals, as well as for principals who withdrew from DLP.
|Available; posted November 2013|
|3B. Third Annual Race to the Top Professional Development Report
During the 2012-13 school year, NCDPI implemented and supported professional development efforts as outlined in the RttT proposal, and regional and local leadership teams implemented a wide variety of professional development strategies. Participants generally agreed that the activities addressed their professional development needs. In addition, some progress has been made on developing local and regional professional development capacity, with more progress expected in Year 4. However, as the State’s educators become more knowledgeable about rapid and significant changes in curriculum standards, student assessments, data systems, educator evaluations, technologies, and overall expectations, they are recognizing that they have a long way to go to integrate all of these changes into their day-to-day professional practices and are reporting a heightened awareness of how much more professional development and support they will need in order to respond to changing expectations and to improve student achievement.
|Available; posted February 2014|
|4. Final Report: Impact|
|4A. Distinguished Leadership in Practice (DLP): Third Annual RttT Evaluation Report – A Final Summary
DLP enrolled 634 principals from 2011-2012 through 2014-2015 from 100 (of 115) Local Education Agencies (LEAs) who were representative of principals across the state. About 20 percent of participants do not complete the rigorous program. A large majority of participants agreed or strongly agreed that the program provided a better understanding of targeted knowledge and skills. There is evidence that participating principals have started to implement knowledge and skills learned in the program. Principals reported increased confidence in instructional leadership and cultural leadership through relationship development; however, there were no statistically significant changes in principals’ evaluation ratings after participation. Participating principals and their teachers agreed that school- and teacher-level focus on student achievement has increased since principals participated in DLP, but principals were hesitant to attribute changes directly to the program. It is too early to establish any direct, causal link between participation and student outcomes.
|Available; posted December 2014|
|4B. North Carolina’s Future-Ready Leadership Program: An Overview of Current Data
Assistant principals in the Future-Ready Leadership (FRL) program take part in six cohort-based face-to-face sessions and engage in assignments, projects, and readings designed to build their capacity as “future-ready” school leaders. In 2012 and 2013, FRL professional development was multifaceted; implementers utilized knowledgeable facilitators to deliver high-quality professional development and provided assignments that required participants to engage their school staff. The complex participant selection process identified participants from over 70 Local Education Agencies (LEAs) and across grade levels. Surveyed participants agreed that: the FRL program was useful; session objectives were clear; the ability to collaborate with peers was beneficial; little needed to be improved for future cohorts; the program helped them build knowledge about effective school leadership strategies and about the North Carolina Standards for School Executives; and they are more likely to encourage a collaborative school environment that targets student outcomes. Participants from the second cohort exhibited statistically significant growth in their NCEES scores after participation. Participants from the first cohort also showed growth, but it was not statistically significant.
|Available; posted December 2014|
|4C. Fourth Annual Race to the Top Professional Development Evaluation Report
Overall, the State has fulfilled the requirements of the RttT professional development plan. Educators are beginning to integrate targeted areas of focus into practice, and there has been growth in local leaders’ professional development capacity. At the state level, NCDPI focused on preparing local leadership teams to provide effective professional development, and on providing them with supportive resources. In 2013-14, participants indicated that this preparation and support met their needs, with NCDPI-RESA sessions receiving the highest ratings. At the local level, Professional Learning Communities were the primary method for translating professional development into practice. Coordinators and principals also reported that the RESAs played a vital role in implementing regional professional development. Efforts to provide quality local professional development have been extensive, but much remains to be done: many teachers reported that they were not yet well-prepared to meet new expectations; disconnects remain between teachers’ and local coordinators’ views of the value of local professional development; qualitative results suggest a decrease in teacher morale and confidence; and local leaders were uncertain how their LEAs would pay for future professional development without RttT support. In sum, much has been accomplished, but effective professional development efforts need to continue, and LEAs will need additional resources and supports to do so.
|Available; posted March 2015|