Reflections on reaching women over the phone in rural India

Data not only defines the world, it has the power to design it. But historically, women have been excluded from policy decisions pertaining to their everyday lives. Researchers aiming to provide accurate representations of populations must ensure that the data being used to shape policies, laws, and legislation represents women’s perspectives. Failure to include women in surveys in India can yield biased results and can potentially lead to policies that adversely affect ~700 million women and girls.

The COVID-19 pandemic disproportionately impacts women. In India, research suggests that women face heightened levels of economic insecurity, food insecurity, and domestic violence during the pandemic and lockdown. These issues highlight the need to include COVID-19’s impacts on women in policy discussions.

Our IDinsight Data on Demand team has been conducting multiple rounds of surveys with 4000+ respondents in eight states in India on COVID-19’s health, economic, and agricultural effects. In these surveys, we aim to reach women 50% of the time to ensure that gender is reflected as a dimension of our understanding of COVID-19. But when conducting phone surveys in countries in which gender roles are more traditional, it becomes vital to consider: Can we reach women over the phone? How?

What are some factors that make it more difficult to reach women over the phone?

In the age of phone surveying, it is tempting to call a household and speak to whoever picks up the phone. After one of our first phone surveys, our Data on Demand team noticed that it was difficult to reach women by phone, likely because of lower female phone ownership and patriarchal social norms that restrict women’s ability to speak freely on the phone.

We analyzed data from phone surveys in which we recorded how the head of household responded to our request to pass the phone to a household member of the opposite gender. We found that men refused to pass the phone to women 11 percentage points (pp) more frequently than women refusing to pass the phone to men. This difference is statistically significant (p<0.01).

Furthermore, other organizations have found that reaching women over the phone can be challenging in terms of reach and privacy. Women are less likely than men to have their mobile phones topped up, particularly during COVID-19 when families face economic hardship.

Due to these challenges, we adjusted our phone survey practices to account for two factors that make it more difficult to reach women: questionnaire framing and surveyor gender.

Lesson 1: Asking “warm-up” questions to a male respondent improves the likelihood he will pass the phone to a female respondent when asked.

During in-person surveys, if surveyors seek to speak to a female household member, the surveyor normally first meets the male head of household, explains the survey to him, and after building rapport, starts surveying the female respondent. Asking “warm-up” questions to male respondents during phone surveys can mimic the flow of an in-person survey.¹

In one multi-round survey, we observed whether or not “warm-up” questions² helped increase our ability to speak to female household members. In round 1, we immediately asked a male respondent to pass the phone to a female respondent if we were aiming to reach a female (we randomly assigned “target genders” to households during sampling). In round 2, we asked the male respondent 4–6 questions before we asked him to pass the phone.³

The graph below shows how male respondents reacted to our request to pass the phone to women. In round 2, when we asked “warm-up” questions, the refusal rate was 4.3 pp lower and the pass rate was 4.1 pp higher than that in round 1. This difference represents an 85% increase in the pass rate when using “warm-up” questions.

Among households we reached in both rounds, we found the increase in pass rates between rounds to be statistically significant (p<0.02). This finding could imply that even households who refused to pass the phone in round 1 were more likely to pass the phone after the “warm-up” questions.^{4 5}

Overall, we believe that these warm-up questions build trust with male household members, reassuring them that we are in fact administering a survey, and not requesting to speak to women for some ulterior purpose.

Lesson 2: Female surveyors are able to complete a higher proportion of surveys with women than their male counterparts.

We experience that female respondents are, in general, more comfortable speaking to female surveyors. Women’s hesitancy to speak to male surveyors is amplified in phone surveys where surveyors are not physically present to show their identification and credentials. In some areas, women receive fraudulent phone calls from unknown men, and they might think that a male surveyor is such a caller. We have observed that our female respondents often ask more questions to male surveyors regarding the survey and the surveyor’s credentials. Finally, female household members may feel pressure from family members to not speak to males on the phone.

To explore how this hesitancy influences completion rates, we compared overall productivity between male and female surveyors. We found that female and male surveyors on average, complete a similar number of surveys each round. In order to understand the effects of surveyor gender on reaching women, we calculated the proportion of surveys where we successfully spoke to women amongst the completed surveys in which we sought to reach women.⁶ We found that the proportion of completed surveys with women was 13.8 pp higher amongst female surveyors as compared to male surveyors. This difference is statistically significant (p<0.02).⁷

While we were unable to control for all factors influencing surveyor productivity (e.g. experience levels), our findings suggest that female surveyors are more successful than male surveyors at completing surveys with female respondents. We recognize that we currently have limited gender diversity on our survey team and are striving to hire more female surveyors, which can improve gender equality, combat existing discrimination, and increase productivity.

Conclusion

Finding strategies to improve how we reach women over the phone is growing in importance as we rely on phone surveys during this COVID period. In our latest survey (in which we intended to reach women half of the time), 37 per cent of our overall respondents were female. Asking warm-up questions and increasing the number of female surveyors we hire are two strategies we are now applying in our work to improve response rates. We also continue to experiment with the best times of day to call a household, structure our questionnaires appropriately, and continue to learn from others’ experiences so that we can survey a greater proportion of women in future surveys. We hope that other researchers and peer organizations will also take similar actions to strive to reach women over the phone.

If any readers have been finding success in reaching women through phone surveys and would like to share what they have found, please comment below — we are keen to learn from your experiences!