New research conducted by IDinsight, funded by GiveWell, explores the preferences and values of individuals and communities in Ghana and Kenya to inform funding allocations
Measuring preferences in Jirapa, Ghana. ©IDinsight/Will Slotznick
IDinsight has worked with Givewell to measure the values and preferences of individuals and communities in order to better allocate development funding. This is one of a series of blog posts on our study. We are interested in discussing these results and are open to collaboration on similar future studies.
Is it better to support a program that increases household income or one that directly saves lives? Decisionmakers in international development, from foundations to governments, frequently weigh complex trade-offs like these between different types of “good” outcomes. Governments in low-and middle-income countries must decide how to allocate funding across different ministries, and funders must decide which sectors, organisations, and interventions to prioritise. The preferences of individuals from low-income communities — those on the receiving end of the hundreds of billions of dollars of development aid ($153 billion from the OECD alone) 1— are rarely brought into these decisions.
Recently, there has been some movement by international development actors towards more participatory methods and programs that aim to capture program recipients’ preferences for specific policies or initiatives. 2 We have only seen a few attempts to capture the preferences for different “good” outcomes and associated moral views, which are required to inform higher-level trade-offs (for example deciding between programs that save lives or increase income).
Over the past two years, IDinsight has partnered with GiveWell to identify methods that elicit people’s preferences in low-income communities and collect data using the best of these methods. This is directly relevant to GiveWell’s decision-making process, which compares the cost‑effectiveness (or amount of “good” done per dollar spent) of charities working in different areas (e.g. health interventions that save lives and cash transfers that increase income). To make these trade-offs, prior to this research, GiveWell staff considered a variety of factors but no data existed on how recipients would make these trade-offs. This blog post shares findings from our recent research collecting data on individual and community preferences in Ghana and Kenya, and details our methodology and challenges with some of these approaches.
During 2018 we conducted extensive piloting in rural Kenya, testing more than 10 different approaches to capture preferences. The aim was to identify and develop methods and questions respondents could understand and capture reliable data of the highest relevance to GiveWell’s decision-making.3 In 2019 we scaled the most reliable methods, interviewing ~2,000 respondents from low-income households in Kenya (Migori and Kilifi Counties) and Ghana (Jirapa and Karaga Districts).
In synthesising the results, we place the greatest weight on the findings that were seen across multiple methods (listed in the following section). There are a number of reasons, discussed in the limitations section, why the central estimates from this study should be more deeply explored. However, we did find that the direction and range of our main findings were relatively consistent across all approaches, which gives us some confidence that we have meaningfully captured preferences. We found that:
Qualitative interviews with respondents give us further confidence in these findings and have been crucial to contextualise preferences and validate results. We integrated qualitative questions into every stage of this research, including: 1)brief qualitative questions in the main survey to validate quantitative answers, 2) focus groups to discuss and debate the quantitative questions with a number of community members, and 3)longer individual qualitative interviews exploring moral reasoning and cognitive processes in more depth. Some of the most important findings:
The findings of this study have played a role in GiveWell changing its moral weights for the 2019 top charities decisions. In its recent update GiveWell has placed substantially more value on programs that save lives (relative to programs that reduce poverty). Additionally, where previously higher weight was placed on averting the deaths of individuals over 5, they have updated to place equal weight on both age groups (under 5 and over 5).
For us, this update by GiveWell provides a proof of principle. It is possible to quantitatively capture preferences on outcomes that can meaningfully inform decision-making. However, this was just one study of a specific target population, in a specific context, where noisy estimates and substantial variation from one region to the next is likely. Practitioners need more studies in different contexts, applying different approaches, and further testing these methods to reduce the uncertainty of these results.
We also recognise that preferences are just one approach to answering these questions. Some argue that decision-makers should be instead aiming to maximise peoples’ subjective wellbeing, which sometimes conflicts with their stated preferences and values. Others may believe we should focus on programs that try to maximise benefit to communities based on more objective data about program recipients’ economic and social contributions. As a global development community, we need to do more exploration to establish the roles and relative importance of these different data sources.
The preferences of populations targeted by aid interventions have not widely influenced funding decisions, in part due to the difficulty of reliably capturing them. We hope this study provides the beginning of a way forward to more readily incorporating their views into these difficult and important trade-offs.
Value of Statistical Life or VSL is a measurement often used by economists to estimate how much people are willing to pay to reduce their risk of dying. While this may seem morbid, the calculation can be helpful for weighing and prioritising different policy agendas using cost-benefit analyses to inform where policymakers should invest.
Our first method captured VSL by asking respondents their stated preference. Respondents were asked about their willingness to pay for a vaccine or medicine (one of the two was randomly selected for each respondent) for themselves or their child (we randomly selected which child under the respondent’s care to ask about, as well as whether we asked about the individual or their child first), that reduces their risk of dying from a hypothetical disease by a small amount (5 in 1,000 or 10 in 1,1000) over the next ten years. Prior to the scenario, respondents complete a series of questions testing for and training their understanding of small probabilities using visual aids.
We conducted two-choice experiments, aiming to understand people’s moral views or their perspectives on how resources should be allocated to achieve different outcomes at the community level.
The first choice experiment presents trade-offs between saving lives and increasing income in the community:
“Program A saves the lives of 6 children aged 0–5 years AND gives $1,000 cash transfers to 5 extremely poor families in your community. Program B saves the lives of 5 children aged 0–5 years AND gives $1,000 cash transfers to [X] extremely poor families in your community. Which one would you choose?”
We varied the value of X, both within and across respondents to capture how respondents trade-off between giving cash transfers to poor families and saving the life of an extra child under 5 across the population. Before presenting the scenarios, we prompted respondents to think about the impact of both types of programs.
The second choice experiment presents trade-offs between saving those in different age groups:
“Program A saves [100/200/300/400/500] lives of people aged [under 5/5–18/19–40/over 40], Program B saves [100/200/300/400/500] lives of people aged [under 5/5–18/19–40/over 40]. Which one would you choose?”
We use the choices made by respondents to estimate their relative value of saving a life in terms of the number of cash transfers, i.e. a monetary value of life from the community perspective, as well as their relative values placed on different age groups.
Our study, especially the VSL component, used state-of-the-art techniques from the current literature and adapted them to local contexts. We conducted extensive piloting to maximise respondent understanding of the questions. We believe it is important to recognise the value of the study — namely being one of the first studies to systematically estimate the preferences among low-income individuals in low-income countries — while accounting for its limitations when applying the results. The two biggest technical limitations to our approach are:
1. While the findings of this study are relatively consistent, the exact estimates vary depending on the framing of the questions asked, and analytical approach.
2. Our methods rely on respondents understanding complex questions, and may be prone to a number of biases
1 March 2019
7 March 2019
2 April 2019
1 May 2019