Report

Impact evaluation: How an accelerated learning program affected over-age early childhood education students’ learning outcomes in Liberia

FasterReading Impact Evaluation: Endline report on over-age ECE student learning outcomes following an accelerated learning program

Michael Sebele , Mtise Mwanza , Jeff McManus , Cassandre Pignon , Leah Mwai , Mico Rudasingwa 12 October 2022

A school in Bomi County, Liberia ©IDinsight

Executive Summary

Rising Academy Network (RAN) is developing an accelerated reading program, FasterReading (FR), to support primary-grade students on foundational reading. During the 2021-22 school year, RAN implemented the FR program in Early Childhood Education (ECE), Grade 3, 4, 5, and 6 primary classes in their schools in Liberia for the first time since the first part of program development was completed. This study focuses on the ECE sections of these schools and specifically over-age children in those classes, who are at particular risk of dropping out.

RAN partnered with IDinsight to generate early-stage insights for program iteration and improvement, and to assess the impact of the FR program for over-age ECE students on learning outcomes and school attendance. We conducted a randomised controlled trial (RCT) in 74 of 95 RAN-supported schools across 10 Liberian counties. RAN rolled out FR in 37 treatment schools from January to July 2022 while 37 other schools received regular ECE programming.

Key findings

The FR program likely led to modest positive effects on reading proficiency, though the precise impact is unclear due to incomplete implementation in half of the treatment schools and partial implementation in a quarter of control schools. We estimate that an over-age ECE student exposed to the full FR program would gain 0.12 more FR reading levels than a student who received none of the FR program (p = 0.10). Given low growth in reading proficiency in the control group, this modest effect represents a relatively more impressive 0.28 SD or 36% improvement over the status quo.
The FR program did not have any adverse effect on students’ math proficiency. The program also did not have significant effects on students’ enjoyment of school or reading.
The FR program had a positive impact on school attendance. Students exposed to the full FR program were 11.4 percentage points more likely to be present for attendance checks by School Performance Managers (SPMs) than students in the control group.
FR students were less likely to practise reading at home. Students exposed to the full FR program were 12.1 percentage points less likely to report practising reading at home compared to students who received none of the program (p 0.05.

Recommendations

The results of this RCT may have been influenced by implementation challenges. Reports from RAN staff and teachers and findings from RAN’s process evaluation in 2021 indicate that the FR program faced several implementation challenges, including teachers’ unfamiliarity with the new curriculum and the need for further training, and insufficient student materials and workbooks. Addressing these challenges may increase the impact of the program. We also encourage RAN to explore opportunities to support students with reading practice at home, and possibly to further tailor instruction to smaller, more homogenous reading groups.

Moreover, some teachers and school leaders were confused about the treatment/control status of schools, since all staff were trained in FR implementation, and the treatment assignment only applied to ECE classrooms. This confusion led to significant exposure of ECE students to the program or to program materials in certain control schools, and incomplete implementation in certain treatment schools. As a result of treatment non-compliance, impact estimates are somewhat less precise than we anticipated in the design.

Despite these challenges in implementation and measurement, FR appears to have a modest impact on reading proficiency. This being the first pilot, the program is in its very nascent stages and RAN plans to iteratively improve on it to attain greater impact. While current impact estimates are imprecise, early results of the FR program suggest that, once implementation challenges have been addressed, the program may be a cost-effective approach to improving reading outcomes.