Q+A: How DataDelta is changing the way social sector leaders drive change

Sarah Lucas is IDinsight’s Global Lead for DataDelta, previously Data on Demand, which aims to equip government and social sector leaders with the highest quality survey data to shape policies and programs that impact people’s lives. She began working on evidence-informed policymaking first at the Center for Global Development, bridging policy research to policy decisions. Then she worked within the U.S. government at the Millennium Challenge Corporation, built to integrate data and evidence into decision-making. Most recently she was at the Hewlett Foundation, supporting non-profit organizations working with African governments to use data and evidence to improve decisions and outcomes for people.

Q. You’ve been in the evidence space for years and have seen first-hand how data-driven insights are integrated into policy decisions. What attracted you to IDinsight?

I’ve worked at the intersection of data, research, and policy for many years. When I was leaving Hewlett, I was filled with lessons about what it takes to do evidence-to-policy work well. The top three were: 1) Engage across methods. No one type of evidence can answer all policy questions. If you are going to be truly demand-driven and decision-relevant, you have to be fluent in a range of methods and agile in matching methods to decisions. 2) Be willing to roll up your sleeves to work with government partners in all the messiness of policy and program decision-making. This includes building trust and thinking about how institutions use evidence over time, not just how individuals (“champions”) use evidence to make discrete decisions. 3) Think about data use and impact from the outset. Thankfully the evidence community is long past the “research uptake plan” that made evidence use and impact an afterthought. But very few organizations think rigorously about potential impact early and use this thinking to decide whether or not to pursue an opportunity. IDinsight embodies all of these lessons so I was delighted to join the team.

Q. What is DataDelta and why did you want to lead this initiative?

What drew me to DataDelta specifically is the focus on people and voice. Much of my work at Hewlett was supporting innovation in the so-called data revolution – taking advantage of new data sources like cell phone data or satellite imagery and new analytical tools like AI to advance development goals. I loved this work. It is important to greatly expand our options for bringing data into social sector decisions. But it also struck me that the data revolution is largely about using innovation to get insights about people without actually talking to people. It can be costly and time-consuming to collect data directly from people, so innovators build all sorts of techniques to learn about people without directly asking. 

I spent my last year at Hewlett focused on issues of data governance and data equity.  These are next-generation data revolution issues, where we still seek the benefits of using data for decisions and also acknowledge the potential harms of doing so. These harms emanate from using data about people without people having a say in it. 

I was looking for a way to stay focused on data-driven decision-making but bring people back into the equation. That’s what DataDelta does. We use innovation to make it easier to hear directly from people, rather than using innovation to go around them.

Social sector decision-makers must understand people’s current needs, preferences, living conditions, and behaviors to create, target, and implement high-impact policies and programs. But collecting survey data is often time-consuming and expensive, doesn’t provide insights quickly enough to inform pressing decisions, and can fail to represent the diversity of the population. DataDelta’s goal is to reduce these barriers by making survey data collection faster and easier, retaining high quality at a large scale. We want a world in which it is easy for social sector decision-makers to hear directly from people before making decisions that affect them.

Q. What’s your opinion on what makes data high-quality, and what difference does the quality of data make when it comes to social sector decision-making?

I think about data quality at three levels. 

The first level is technical. This is everything we build into our field operations and data collection systems that ensure quality. This includes extensive training of enumerators and field supervisors and lots of in-person, real-time feedback to correct field-level errors as soon as they occur. It includes high-frequency checks to confirm the logical consistency of responses and flag errors by detecting outliers, high non-response or refusal rates, speed violations, etc. Depending on the survey it might include in-person or phone backchecks, audio audits, or specific checks that we tailor to a given survey. DataDelta prides itself on high technical standards for data quality because data-informed decisions are only as good as the veracity of the data behind them. 

The second level is relevance. Data that is technically rigorous is only “high-quality” for decision-making if it is tailored to decisions; if it is available in time to inform decisions; and if it accurately reflects the people decision-makers really need to see. DataDelta’s goal is to make large-scale survey data decision-relevant. We do this by first, deeply engaging with partners to tailor surveys to their specific decision-making needs. Second, using our custom-built software, SurveyStream, to accelerate survey management and data quality checks so that we can deliver data when they need it. Third, we develop sampling innovations that allow us to represent both large populations and sub-groups in a way that doesn’t break the bank. This is particularly important because many social sector programs and policies focus on subgroups (e.g. children, women, and people at a given income level) so being able to see these people in your data is essential. It is also key to using data to advance gender equity. 

Data quality is ultimately about voice, equity, and justice.

The third level is equity. Data quality is ultimately about voice, equity, and justice. It is about creating a direct pipeline from the minds and hearts of people to the desks of technocrats. We focus on technical data quality to protect the integrity and “truth” of what respondents tell us – deeply intimate things about their lives, well-being, health, aspirations, and experiences with the government. To do justice to their aspirations and their trust in us, we need to carry this information through the data value chain in a way that protects its integrity and truth. 

For example, in our work with the Aspirational Districts Program in India, we are using DataDelta (formerly Data on Demand) survey data to verify administrative data collected at the district level. The analysis has been startling – so much gets mangled or mistaken as it travels from respondents or facilities to the data dashboards that inform decision-makers. Yet for decades consequential policy decisions have been made using this wrong information. To me, that feels like a genuine issue of justice and power. Seeing what it takes to collect data from people, and what it takes to safeguard its authenticity until it reaches decision-makers’ desks, feels like we are chipping away at power asymmetries and injustice. In the sea of digital data (much of which is important and useful!), it feels extra important to bring data collected directly from people about their lives and experiences, with their consent, to bear on policies and programs.

Q. With the world advancing into more technology-driven solutions, social sector leaders are increasingly gaining access to multiple sources of data. Do you think primary/survey data has a place in the future of decision-making? What role will it play?

Despite the rise of secondary data sources, primary data continues to play a pivotal role in social sector decision-making. In fact, nearly a third of Sustainable Development Goals rely on household surveys for data to monitor progress.¹ There are some things you just can’t know unless you ask people. Primary surveys are essential for understanding intra-household dynamics, for understanding differential impacts of macro phenomena (like pandemics or weather disasters) across vulnerable sub-groups, and for understanding people’s preferences that are asserted (rather than inferred, for example, through social media). Household surveys remain vital for understanding poverty, revealing inequities, and targeting social protection services. They are even critical for making more novel data sources accurate. Primary data are important to train and ground-truth AI models and remote sensing data. Despite their high value, in many places, survey data remain out of reach for routine decision-making for the reasons mentioned above. The global survey community recognizes that innovation is necessary if we want this high-value data to serve us into the next decade. The DataDelta team is part of that innovation. We are excited to be part of a broader community that is lowering barriers to hearing directly from people and improving the policies and programs that affect their lives.

Q. This IDinsight initiative was previously called “Data on Demand”. Why did you change the name to “DataDelta”?

“Delta” means change. We wanted a name that speaks to the purpose of our work – to create positive social change by making it easier for people in power to understand and meet the needs of the people they serve. One of our superpowers is being tailored to the decision-making needs of social sector leaders – offering bespoke data at a large scale. This motivated our earlier name, “On Demand.” But asking for the data isn’t enough. Leaders need to use it to inform their decisions, improve their policies and programs, and impact people’s lives. That is what we are all about. 

We also like “delta” because it refers to river deltas which are “the source” for watersheds, rich nutrients, and forward movement. Just as we go straight to “the source” to gather data.  It also makes a beautiful pair with our custom-built survey management software, SurveyStream.

Q. What is your overall vision for DataDelta? What should we expect from your team in the next 12 months?

Our overall vision for DataDelta is aligned with IDinsight’s mission – to improve lives through data and evidence. Our contribution to this vision is enabling social sector leaders to always have data about people, directly from people, at their fingertips when making decisions that will affect people. 

DataDelta equips government and social sector leaders with the highest quality data to shape policies, respond to crises, and make more informed decisions that impact people’s lives. This can take several forms – informing discrete decisions, strengthening existing data systems, or validating other data types. Read more about our existing projects in our news announcement.

As I celebrate my one-year anniversary in the IDinsight Nairobi office and one-year working across a global DataDelta team, I am really proud of how far we have come. I am also excited about what the next year will bring. Over the next twelve months, we will greatly expand our work to support social sector leaders and to strengthen public sector systems for data-driven decision-making across India, the Philippines, Kenya, and perhaps beyond. What a thrill it is to lead a team and a program that so fully embodies the evidence-to-policy lessons I came in with, and that teach me more every day. Stay tuned for more about what we are learning and achieving with DataDelta in the months to come.

Read more about DataDelta and get in touch with our team.