Using digital maps to collect high-quality data in rural Zambia

Organizations and policymakers looking to support people living in poverty often rely on community and household data to identify issues and design effective programs.

But in rural areas of low-income countries, collecting accurate and timely data is difficult and expensive.

IDinsight is piloting a project called Nano in a rural Zambian chiefdom to collect high-quality data for community leaders that will inform their decision-making. Nano aims to use a lean data collection system that is largely automated and managed remotely, which provides a cheaper, quicker alternative to traditional survey approaches.

Using digital maps to survey rural communities

For a sample of households to be representative of the larger population, a complete and accurate sampling frame is required — essentially, a full list of units (e.g., households/villages) from which to choose the sample. But in many low-income countries, reliable village and household listings may not be available. This was the case in the Zambian Chiefdom.

Thus, we had to create our own sampling frame. We worked with digital maps of the Chiefdom and an existing dataset with GPS coordinates of rooftops in Zambia (obtained from The Humanitarian Data Exchange[footnote]The Humanitarian Data Exchange. (2019, April 24). HOTOSM Zambia Buildings. Retrieved from The Humanitarian Data Exchange website[/footnote]) to partition the Chiefdom into small non-overlapping geographical units. We call these units enumeration areas, abbreviated as EAs.

Next, we randomly selected a fraction of the EAs to be a part of the study. During data collection, surveyors were instructed to survey households only in the selected EAs.

Maximizing surveyors’ effectiveness

Any sampling strategy is only as good as the surveyors’ adherence to it. Thus, we created multiple tools to help the surveyors survey all households in a given EA, and avoid households outside of it. These tools are largely automated and can be monitored remotely.

We trained surveyors to use a mapping app (Avenza Maps) with preloaded EA shapes (Figure 1). The grey target is the surveyor’s location, while the grey lines are EA borders. When enumerators approach a household, they view their location against the map to check whether the household is within their assigned EA.

Figure 1: Avenza Maps’ smartphone interface

We also monitor surveyors remotely to ensure that they only visit households in their EA. Figure 2 demonstrates this monitoring system. Each colored dot shows a household that has been surveyed. We used this analysis to notify surveyors via text message when they are surveying outside of their EA. In our data analysis, we excluded households located more than 20 meters outside of an EA.

Figure 2: Map household survey location against EA boundaries

To ensure that all households in an EA are surveyed, we used the rooftop data to direct surveyors to unvisited areas with high probabilities of households. We did this by sending surveyors a message (shown in Figure 3) with the detected rooftops (grey dots), households that they had already visited (blue dots) and areas to visit (red arrows).

Figure 3: Map directing enumerators to unvisited areas of an EA

Remote oversight

Remote supervision of surveyors included a comprehensive evaluation of their performance with timely feedback and bonus payments to incentivize high-quality data collection.

High-frequency checks (HFC) are a standard way to track surveyors’ progress and quality in real-time [footnote]Boyer, C. (2019, April 26). Background. Retrieved from PovertyAction High Frequency Checks [/footnote]. HFCs run daily throughout data collection and track how many households each surveyor visits, how long each survey takes, and their compliance with protocols. Surveyors whose indicators are statistically different from the team averages are flagged and given feedback.

We have taken HFCs one step further by automatically sending supervisors and the IDinsight project team messages about a surveyor’s performance with instructions on how to deliver targeted feedback.

Figure 4 presents a table the IDinsight Nano team received with surveyor progress. Cells highlighted in yellow signify that the surveyor average differed from the total average, which informed IDinsight on each surveyor’s relative performance.

Figure 4: Message to the IDinsight team reporting surveyor performance

We also use the HFC results to incentivize surveyors to follow the protocols and improve their performance with bonuses. The bonus structure was carefully developed, using a group of indicators that could be measured accurately and which balanced survey quantity and quality. The bonus payments were dispersed via surveyors’ mobile money accounts to avoid travel costs.

Lastly, using a behavioral economics approach, we delivered a set of automated text messages with “good practice” nudges to remind the surveyors about important data collection protocols, such as “Remember to only visit households within you EA” and “Read the consent form slowly and clearly.”

Looking ahead

As we complete this pilot project, we hope to gain further insight into how to collect high-quality data at a lower cost in rural areas. We recognize that some of these tools may be useful to other practitioners, although some may not. These strategies are also context-specific: IDinsight strives for high-quality data in all our projects, but the tools we use vary based on available data and the information we want to collect. We look forward to sharing more of our findings as this project evolves.

This project is funded by the Global Innovation Fund.

The Nano project falls under IDinsight’s Innovations workstream. Over the coming weeks and months, we’ll share more about how our teams are innovating in other areas, such as machine learning and results-based financing. To learn more about IDinsight’s Innovations projects, click here.