Walk the bottom line: tailoring impact evaluations to benefit business

In this blog post we discuss our experiences collaborating with for-profit agribusinesses to evaluate an initiative promoting poultry rearing for smallholder farmers across East and West Africa — and what we learned about tailoring impact evaluation tools to work with the private sector. We recently concluded baseline surveys and we will share more results and reflections in the coming year.

Why work with agribusinesses to reduce smallholder poverty?

Smallholder farmers account for 60% of Africa’s workforce, 40% of whom are women. In populous African countries, small-scale farmers produce the bulk of domestic agricultural output, yet the majority live on less than US $2.00 per day.

Agribusiness that enhances the productivity of smallholders has the potential to grow Africa’s agricultural sector while contributing to poverty reduction efforts. Yet the evidence on how agribusinesses impact farmer profits is mixed so far. Policymakers and development partners face a key question: how can agribusinesses work with smallholder farmers to improve livelihoods while still making a profit?

To contribute to this knowledge base, IDinsight joined an initiative to evaluate a corporate model where agribusinesses market fast-growing chickens to smallholder farmers in East and West Africa.¹ While the model may be profitable for the suppliers, our role is to determine whether smallholder families also see improvements in welfare. To do so, we designed a series of impact evaluations to measure changes in revenue, family nutrition, and women’s decision making power for over 2,000 farmers in more than 250 communities across two countries, over the course of three years.

Relative to our NGO and government partnerships, conducting a successful impact evaluation with for-profit agribusinesses requires an even more tailored and dynamic approach. In this post, we highlight three lessons on how to adapt impact evaluations to collaborate with the private sector. We hope these lessons can serve as useful guideposts for other research teams exploring the intersection of agribusiness and development.

Lesson 1: Maintain flexibility in your evaluation design

Governments and NGOs often take time to create inclusive processes and refine their activities based on the interests of many different stakeholders. In contrast, private companies prioritize serving and expanding their customer base, and can act quickly and flexibly to respond to market dynamics without needing to consult voters or donors. For example, one agribusiness we work with found that distribution sites were too scattered across a particular region, limiting its ability to maintain a consistent supply of chickens for sale. To increase efficiency, it decided to focus its expansion on a more limited area that might be easier and more economical to serve. These market-driven adjustments, while logical for a business, at times posed serious challenges to maintaining the technical rigor of our impact evaluations.

In response, we adapted our approach in two important ways:

• Design: So the companies would have sufficient flexibility to target customers, we decided to pursue a quasi-experimental design that did not randomly determine the areas into which the companies would expand. Rather, we worked with their field teams to identify and vet a long list of potential communities for chicken sales. Based on this list of viable treatment communities, we developed a list of similar comparison communities.² Both lists contained sufficient replacement communities so that if companies struggled to stock or sell chickens in one location, they could easily redirect their expansion efforts to one of the previously vetted replacements. This process allowed the companies to adapt to market circumstances while ensuring we reached the required number of study communities.
• Sampling: After companies began marketing chickens in the selected treatment communities, we planned to survey customers within six weeks after they purchased chickens.³ In theory, this approach would have enabled us to focus on smallholders who actually reared the chickens and to find suitable matches for those customers from the comparison communities. In practice, it was difficult to reliably plan when local distributors would stock chickens and sell them to farmers. In response, we pivoted to a prospective sampling approach where we over-sampled farmers likely to buy chickens, based on their stated interest at marketing events.⁴ While this ex-ante approach increased our surveying costs at baseline, it also enabled chicken sales to occur organically and allowed us to provide companies with valuable lists of potential customers.

Lesson 2: Invest in building trust

Data privacy is especially important in the private sector, in part because customer information can give a business its competitive edge. It was really important for us to take time to build trust with our corporate counterparts, so they knew how we would safeguard their data and protect their market competitiveness.

Our collective efforts to meet face-to-face layout our respective goals, and directly exchange information were critical to building relationships that enabled our research. We found it useful to:

• Clearly demonstrate how we would use company data for our research
• Involve several leaders in making evaluation design choices based on that data
• Agree on data access as early as possible, to avoid future delays or surprises
• Use formal protocols on data privacy, such as non-disclosure agreements
• Share regular updates on our work so counterparts were never caught off guard

Lesson 3: Provide valuable business insights to companies every step of the way

Our ability to measure the social impact of this for-profit model depends on the companies’ willingness to adhere to the evaluation design — essentially, to market chickens in treatment communities and not in control communities.

To maintain buy-in and create value for the companies even before we know the final results, we worked to collect and share business and operational insights that speak to the companies’ bottom line.

• Targeted marketing: We considered the demographic (age, gender, wealth, and education levels) most likely to purchase chickens and explored the primary marketing channels influencing farmers to buy them. Companies will be able to use these insights to assess where demand for chickens is highest and what marketing messages could resonate most with potential customers.
• Customer retention: It is crucial to the companies’ bottom line that customers not only buy chickens once but keep buying over time. To help companies understand and respond to trends in customer satisfaction, we outlined perceived likes and dislikes about the chickens and measured what proportion of customers would recommend the product to others. We will continue collecting this data as we track chicken sales over the next two years.

Whenever possible we aim to pair social impact findings with a discussion of markets and customers to keep the companies engaged in our research and pave the way for subsequent phases of the impact evaluations.

Tailoring impact evaluations to the business bottom line

Agribusinesses are important partners in agricultural development efforts and there is more to learn about how they can also positively contribute to smallholder farmer livelihoods. As we continue to work on this and other market-based initiatives, we hope to further refine how we tailor our impact evaluations to ensure that measuring social impact can result in more inclusive outcomes while also helping a company’s bottom line.