GiveWell: New Incentives
IDinsight is running a large-scale decision focused evaluation of a Conditional Cash Transfer (CCT) program that seeks to incentivize routine immunization in three states in North West Nigeria – a context where immunization coverage is extremely low and a high proportion of under-five deaths are attributed to vaccine‑preventable diseases. The program is being implemented by GiveWell incubation grantee New Incentives, and IDinsight’s primary role is that of evaluator.
Evidence from the evaluation is expected to inform GiveWell’s decision of whether to designate New Incentives as a Top Charity in 2020 and whether to recommend that donors allocate funds to New Incentives. These decisions could unlock more than US$20 million in funding to allow New Incentives to scale up its program across Northern Nigeria. In the period leading up to the final data collection, IDinsight is making evidence-based recommendations to New Incentives that aim to improve its program and increase its impact.
The key IDinsight service on this project is implementing a clustered randomized controlled trial (RCT) to inform GiveWell decision making. IDinsight also works closely with New Incentives, providing evidence to inform program decisions and helping New Incentives improve its monitoring processes and use of data. For example, IDinsight conducted a separate RCT before the main study began in order to test the effectiveness of different incentive sizes.
Also prior to the main study, IDinsight conducted a baseline survey in much of the study area, gathering data on current immunization coverage and demographic, geographic, and economic characteristics. During the study period, IDinsight conducted a quality assessment of satellite-derived population estimates and developed a protocol by which New Incentives staff can use such data to estimate the number of unvaccinated-but-eligible infants in relevant communities.
The decision focused evaluation results are expected by June 2020.
IDinsight’s incentive RCT found that increasing incentive amounts above a certain size had minimal influence on their effectiveness. Meanwhile, evidence from the baseline demonstrated vaccination coverage in study areas was low and that lack of awareness and low motivation were the most-frequently cited reasons.
IDinsight’s data quality analysis found that satellite-imagery-derived population estimates, generated at limited cost, can improve on existing sources in terms of accuracy. New Incentives has hired new staff that will specialize in implementing the satellite‑data estimation protocol.
In preparation for the RCT, New Incentives used IDinsight evidence and recommendations to refine its incentive sizes, optimizing for cost and effectiveness. New Incentives then used IDinsight’s baseline data to refine its implementation plan – ensuring it addresses awareness as a potential barrier to increasing coverage. During the RCT period, IDinsight’s population estimation protocol is helping New Incentives determine where the most potential beneficiaries live and redirect resources to those areas.