How to identify the best length and time for a phone survey

This post is aimed at research practitioners experienced at administering in-person surveys and interested in beginning phone surveys.

After operationalizing a callback procedure and a tracking system that helped us increase respondent reach, we began to analyze data on-call times (automatically calculated in SurveyCTO as a variable entitled ‘start time) and survey length (automatically calculated in SurveyCTO as a variable entitled ‘duration’) to ascertain if there were any additional ways to reach more respondents. Our findings are below:

Call times: not always bright and early

In our context, most surveys were completed in the mornings. In the first two pilots, field officers had discretion to call, and for the latter pilots, we equally distributed call slots. As the graph illustrates, in our context, most surveys were completed in the morning. However, since our phone surveys were conducted in the winter, field teams expressed more members from more agricultural households were available in the latter half of the morning since they were not tending to crops in the early morning.

We hope to continue monitoring when calls are completed to see if there are seasonal patterns we should be aware of. The data on-call times helps us inform callback protocols to ensure surveyors are calling respondents at the most optimal times.

Survey length: keep it reasonable

In-person surveys sometimes take hours, but how long would respondents stay on the phone? The literature is quite mixed on the appropriate length of surveys, but reaches a consensus that phone surveys should be shorter than in-person surveys.

Understanding the optimal survey duration is extremely important for two reasons:

1. Since we introduce the survey by asking the respondent for X minutes of their time, it is important that the expected survey time is reasonable. If the respondent feels it is unreasonable, they are more likely to refuse.
2. Understanding survey length helps account for respondent fatigue. The quality of the responses might decrease if respondents feel agitated or tired from being on the phone.

The third consideration in some contexts may be the cost of phone plan/reimbursements for respondents. In our case, respondents pay for their own phone plans, but in India, phone penetration is very high and airtime is much cheaper compared to other countries.

In the first three pilots, we were primarily interested in understanding how to ensure most respondents picked up the phone, so we did not invest significant time in developing questionnaires. Instead, we asked a few simple questions sourced from teammates. We sought to test optimal survey length by implementing an experiment in two of our pilots.

We constructed a questionnaire with three sections of the same approximate length and randomized how many sections each respondent was tasked to answer. After introducing the survey and asking for consent, this would mean that surveyors were administering a survey of different lengths. We collected data on the appropriate survey lengths and found that the ideal survey in our context was roughly 10–15 minutes. Our surveyors also consistently noted that around the ten-minute mark, respondents began becoming more agitated and started asking surveyors how much longer the survey would last.

Of the phone surveys we have completed that were not refusals, the majority were completed in 10–15 minutes. From all of our pilots, we have not yet been able to consistently reach respondents for longer than 20 minutes.

We continue experimenting with survey length to build our knowledge of optimal survey duration and improve our understanding of the contexts in which longer phone surveys may be possible. For example, the permissible survey length (without causing fatigue) could be negatively correlated with the mental effort required for participating and could be increased through the use of incentives.

Building out phone calling protocols with an understanding of the optimal call times and adjusting survey length for the optimal duration can help increase respondent reach. We will continue to collect data on call times and duration to understand how to best reach respondents in India.

If you’re experimenting with phone surveys or have feedback about our processes, please comment below or on social media, or reach out to our team, abhilash.biswas@idinsight.org and mitali.mathur@idinsight.org, with your thoughts and suggestions.