Part three in our series of lessons learned based on a 6,000 person survey in India.
Field Manager Lipika Biswal prepares slides at a Data on Demand training last September.
Many governments and other organizations are implementing large-scale phone surveys during COVID-19. In our first post, we provided information on hiring; and in our second, we described some of the steps we took to ensure high-quality data. But what are some of the things teams need to do every day to keep a phone survey running smoothly? In this post, we go over some of the daily practices we’ve implemented.
During a typical day of phone surveying, a surveyor will receive assignments and make phone calls, communicate with the rest of the team, and debrief with their supervisors and team. Here, we introduce our practices and protocols to make sure that each part of the day runs smoothly.
For our phone survey, we used a callback protocol in which surveyors made calls during three different slots throughout the day (morning, afternoon, and evening) to optimize reach. In our first round, before each calling slot, a surveyor would receive an email assignment for their upcoming slot from their District Coordinator (who was able to use the tracker on Google Sheets to filter which assignments were forthcoming). The assignments included households to call for the first time, households not previously reached to try to call again, and households who had made an appointment and requested to be called during that specific time slot. Surveyors would message their District Coordinator to confirm receipt of their email assignment and ask questions. Surveyors would then proceed with their assignments while submitting survey forms upon completion. After the first round, we worked with our Data on Demand data systems team to create an automated email system for assignments. In our second round, we used this automatic email system to send out assignments, which allowed District Coordinators to prioritize their time monitoring productivity and data quality.
We used WhatsApp as our primary method of communication between our field teams. We established organized WhatsApp groups to ensure that messages were appropriately streamlined so that all queries could be answered. The following WhatsApp groups were used on a daily basis:
Through these channels, we were able to establish appropriate feedback loops to directly communicate updates, reminders, and important announcements with field teams while simultaneously receiving feedback about challenges, concerns, and questions from surveyors. In addition to these WhatsApp groups, we used a concern clearing sheet: District Coordinators could quickly record any questions on this sheet. Our team would then respond to these concerns and share the answers in the group chats to disseminate pertinent reminders and solutions quickly.
Debriefs between District Coordinators and surveyors have been essential to discussing the challenges and questions surveyors face. Our District Coordinators would debrief with their surveyors every evening after data collection, then speak to their State Coordinators, who later spoke to the IDinsight team. Given our short data collection timelines, it was essential to establish a set cadence of debriefs so that we could take immediate action on any important takeaways or decisions.
During these debriefs, we aimed to cover four main topics:
After each debrief, our team was able to take actions to help boost productivity, answer concerns, and ease any burdens on surveyors. We were able to understand the reasons surveyors may not be able to complete surveys (phone charging challenges, network outages due to weather, etc.) and appropriately adjust call assignments. We were also able to isolate challenging survey questions or potential data quality concerns. By answering queries on these questions and protocols, District Coordinators were able to send updates to their surveyors emphasizing reminders and clarifications.
5 June 2020
9 November 2020
22 October 2020
30 March 2020