A look into results of the Village Enterprise Development Impact Bond evaluation

COVID-19 disrupted lives and livelihoods, especially of the most vulnerable people. Social protection programs, like a multifaceted poverty graduation package of cash, social and health support, have been shown to improve the livelihoods of people living in extreme poverty. Evidence shows that these programs best support households when offered as an integrated package, compared to one component like cash transfers only (Banerjee et al., 2020).  From 2017-2021, IDinsight conducted a randomized controlled trial (RCT) to assess the causal impact of the Village Enterprise graduation program on household consumption and asset value in Kenya and Uganda. This RCT is part of a $5.32M Development Impact Bond (DIB), the first DIB launched in Sub-Saharan Africa, and the results of the RCT were to determine outcome payments made from outcome payers to the service provider (read more about the Village Enterprise DIB here). This project was interrupted by the COVID-19 pandemic, but we now have results from the evaluation and also some findings that point to how households did or did not remain resilient with Village Enterprise program support during COVID-19.

The Village Enterprise program

Village Enterprise is a non-profit organization that aims to end extreme poverty in rural Africa through innovation and entrepreneurship. They target communities in extreme poverty and offer them an integrated package, a one-year program that includes training on business skills and financial literacy, mentorship, and seed capital to start small, sustainable businesses and form savings groups.

Village Enterprise, together with Instiglio and outcome payers designed the first Development Impact Bond in Sub-Saharan Africa to finance the graduation model. From this, Village Enterprise implemented a graduation program in western Kenya and eastern Uganda, in which they offered 14,772 ultra-poor households across 241 villages to participate in the program. The program was rolled out in seven cohorts from 2017 to 2020, and the participants were organized into business groups of three entrepreneurs.

IDinsight was the independent outcomes evaluator; we designed and conducted an RCT to measure the causal effect of the program on beneficiary households’ consumption and net assets value. The endline data was supposed to be collected in two rounds, in Spring 2020 and Spring 2021, but due to the COVID-19 pandemic, the exercise was conducted in person in May-August 2021. This was 6 months to 2.5 years after the implementation of the one-year program (depending on when each cohort finished the program). IDinsight also oversaw the verification of the cash transfers to all seven cohorts.

Main findings

Our findings are consistent with other evidence showing poverty graduation programs have positively impacted the livelihoods of those living in extreme poverty (Bandiera et al 2017, Banerjee et al 2015, Blattman et al 2016, Gobin et al 2017, Karimli et al 2019, Sedlmayr et al 2020). The results of the study show that the Village Enterprise program had a positive impact on household consumption and net assets despite the pandemic. On average, those who were offered the program consumed 9.9 USD (or 6.3%) more per month and they had USD 40.5 more in net assets than the control group in both countries. Effect sizes for both outcomes were notably larger in Kenya. Some of the differences that we think could have resulted in larger effect sizes in Kenya are baseline wealth and grant size amount. The Kenya sample had higher baseline wealth levels which are positively correlated with consumption effects. Some households in Kenya also received larger grant amounts compared to the regular cash transfer given to Uganda households.

The key drivers of the consumption effect are food expenditures, with green maize, beef, dried and smoked fish, maize grain or flour, and chicken having the largest effects for Kenya. For Uganda, treatment households consumed significantly more chicken, tomatoes, dried and smoked fish. Treatment effects on assets are driven more by increases in household assets, with Ugandan households spending more on livestock assets while Kenyan households spent more on new roofing, new homes, and durable assets.

We also analyzed the outcome effects by key subgroups to determine how the impact was correlated with conditions and characteristics of the households. In Kenya, some households were offered larger grants ($150 per household) while others were offered small grants ($50 per household) so we could assess whether different grant sizes influenced the effect of the program. We find that households that received larger cash grants had significantly larger treatment effects for assets (87.7 USD for larger grants vs 5.9 USD for small grants) but they did not have significantly different treatment effects for consumption compared to those that received the small grant. We also find that treatment effects on consumption were generally larger for households that had more wealth prior to the introduction of the program, though there was no strong correlation between baseline wealth and program effects on assets.

The results of our study also reinforce research showing that the impact of multifaceted programs is sustained over time and may make households resilient to economic shocks. The Village Enterprise program was rolled out in cohorts and our study found that earlier cohorts had similar or larger impact estimates than later cohorts for consumption effects, though the impact on assets over time is less clear.

Additionally, we collected self-reported data on the impact of COVID-19 on households’ well-being.  Most (87%) respondents reported that the COVID pandemic affected their economic well-being negatively. However, households that reported being negatively impacted by the pandemic had similar treatment effects as households that reported positive or no impact. This speaks to the potential for poverty graduation programs to be an important tool for policymakers to deploy to make ultra-poor households more resilient to economic shocks, like COVID-19.

IDinsight’s findings from this evaluation contribute to the breadth of evidence on the impact of graduation-style programs on the welfare of people living in poverty and can inform those implementing similar poverty alleviation programs. Our research shows the important role that mentorship and business skills training, in addition to cash transfers, play in efforts to build sustainable solutions out of poverty. Our findings show that, in spite of the pandemic, beneficiaries of the Village Enterprise program were able to increase and sustain household consumption and assets. As a result of this impressive impact, Village Enterprise achieved the maximum outcome payment of the DIB. We congratulate Village Enterprise on their hard work, and we look forward to the scale-up of their program and other poverty graduation programs in the region.

Results reveal – A note from the team
(Watch the video)

Our team did something unique to learn about the impact evaluation results. For most evaluations, the research team does data cleaning and analysis for several weeks or months, and through this process gradually learns what the results will be, though those results jump around as the analytical code gets updated and finalized. Typically, the person leading data cleaning/analysis will learn about the results before others on the team.

We thought that it would be fun to finalize the cleaning and analysis code before finding anything out about the results, and then run the analysis code for the first time on a live call to learn the results together. The main reason for doing this was to have a fun and inclusive team-building activity that symbolized the culmination of four years of hard work as a team. It also increased the excitement factor for us since we knew that the results would not jump around with further cleaning/analysis.

As you’ll see in the video, the way that we were able to accomplish this was by simulating the treatment assignment again and doing all of the cleaning and analysis using our ‘fake’ treatment variable, then on the live call switching out the fake with the real treatment variable and running the final analysis code. We invited current IDinsighters working on the VE DIB project, as well as alumni to the call although not all are captured in this video.