Impact evaluation of a school leadership development programme in Western Cape, South Africa

Key takeaways

We found low compliance rates with the ILDP, with 56% of treatment schools completing the ILDP and only 33% regularly attending programme activities. Compared to better-resourced schools, we found that lower-resourced schools were more likely to complete the programme, suggesting that lower-resourced schools might find participating in the ILDP particularly beneficial. Low programme completion rates were driven by a variety of factors, including financial constraints, personal reasons, not getting promoted or lack of school or supervisor support.

For schools that completed the programme, we find evidence that some ILDP practices were implemented. Specifically, ILDP schools are more likely to follow best practices on strategic planning, have better systems to monitor the implementation of changes and personalization of instructional plans, and leaders are more likely to deliver feedback to teachers via direct conversations.

There were modest improvements in school management practices between schools that were and were not offered the programme (average treatment effect estimate – ATE). The control and treatment mean of management practices measure was 2.97 and 3.125, respectively (difference 0.155, p-value=0.103). This represents an about 5% difference between the two groups. Adjusting for non-universal participation (with only 56% of the treatment group participating in the programme), the treatment of the treated estimate (TOT) was 0.279 (p-value=0.068), representing a 9.3% change compared to the control group. Both estimates provide valuable insights into the programme’s impacts. The ATE is useful for comparing ILDP impacts with other programmes aimed at enhancing school management practices, as it includes schools that choose not to participate but are still within the population of interest. On the other hand, the TOT estimate assesses the impact on the specific group of schools that are interested in and able to participate in an intensive leadership development programme like the ILDP.

We did not find any impacts on either the quality of teaching or student test scores. We attribute the lack of impact to a few possible reasons. First, the sample size was likely insufficient to detect impacts on downstream outcomes. Second, participants and ILDP implementers suggested that more time might be needed for the diffusion of changes due to the long programme causal chain and gradual implementation. Qualitative findings indicated positive changes in school culture, including improved student discipline, classroom management, and teaching quality. However, these changes were often limited to specific grades or phases and lacked consistency across teachers, possibly preventing school-wide effects. It is possible that improvements occurred in the classes that consistently implemented the new techniques.