Tackling gender inequality through improved data quality

The Global Gender Gap Report of 2022 indicates that at the current pace, it will take us 132 years to reach full gender parity (World Economic Forum, 2022)¹. No woman, currently alive, will ever experience a world where she has equal rights, resources, opportunities, and protections to those of their male counterparts. Additionally, the scarcity of data on gender specific-challenges leads to immense oversights at the policy level. 

The data we do have on gender illustrates a dire situation. As of 2021, nearly one in five women enters marriage before age 18.² A woman in a car crash is 47 percent more likely to suffer severe injuries than a man, primarily due to male-tailored car safety features.³ Women take up a share of 28 percent of managerial positions worldwide, which has not increased between 2019 and 2020.² At least once in their lifetime, one in four of ever-partnered women (over the age of 15) suffer from physical or sexual violence inflicted by their partners/husbands. And across media, for every woman character, there are 2.24 men characters.⁵ 

Data is the starting point for understanding the gender-skewed reality and implementing evidence-based policies to address it. The quantity of collected data matters – but the quality is as important. Data quality defines the conclusions and recommendations we draw from our studies.

If the data misrepresents ground realities, any and all policies will be ill-equipped to tackle pervasive challenges.

Hurdles in data quality

Challenges linked to data quality vary across contexts, methodologies, and research topics. For more accurate documentation of women’s voices, tailored survey tools have been developed, including gender-specific indicators, frameworks for assessing women’s empowerment, and vignette-framed questions capturing attitudes toward gender norms (for example, by World Bank, Oxfam, or WHO). 

As part of field data collection activities, most survey teams strive to maintain high data quality and rigor through spot-check forms, backcheck calls, high-frequency checks, audio audits, and daily debriefs with enumerators. Nevertheless, data quality challenges still emerge. The setting of the interview, questionnaire administration, enumerators’ biases, and preparedness have a direct impact on the quality of recorded data. Beyond that, data quality might also be affected by the respondent’s gender.

I observed these hurdles as part of my data collection work for a project focused on agriculture and farmers’ collectives. It gave me the opportunity to first-hand observe these nuanced challenges.

Why would data collected from women be more polluted? 

Behind every data point lies the human experience of conducting and participating in a survey. These experiences can substantially differ for men and women. 

I learned this through my data collection journey. Being present for numerous surveys carried out in Telugu (while not speaking Telugu) led to me observing participants’  interviews, surroundings, and behaviors. I noted striking differences between women’s and men’s participation in surveys. I realized understanding behavioral differences is essential to tackling gender-specific data quality challenges.  

Frequently, women weren’t fully available. When we arrived at a respondent’s household, women were often busy with childcare or household chores. As we conducted our survey, their lives, and tasks continued in parallel. Children often accompanied their mothers throughout the 40-minute-long conversations, asking for attention and interaction. Other times, women continued peeling vegetables, weaving baskets, or periodically looking back into the kitchen to continue the housework we interrupted. When we surveyed men, children were barely ever-present, and men were not disturbed by household duties. Women were more distracted and less likely to think through their answers and hence, explain their preferences and the challenges they face.

Intra-household dynamics sometimes determined who answered our questions. Upon identifying the household, we asked for the correct respondent – in cases where it was a woman, often the whole family gathered in one room. Men present in the room spoke and answered questions unprompted. They interrupted women, suggested answers they should say, or continuously repeated, “She doesn’t know anything”. These disruptions in the interviews led to women retreating and being less willing to share their opinions. The surveys would go on, but the responses were men’s – husbands’ words repeated by a woman. If women’s data points come from men, how can we know what women think, prefer, and want? 

Women were not comfortable being interviewed. Some women did not make eye contact, fidgeted, and did not engage fully in the conversation. The evidence is mixed; however, some studies conclude that women are less likely to trust a stranger than men.⁶⁷ Enumerators play a key role in defining the dynamics of the survey. In instances where surveyors spoke to respondents for a few minutes prior to the survey, participants appeared more comfortable during the survey itself. If women don’t trust nor feel at ease with the enumerator, we could be missing essential data points for a holistic understanding of their realities. As Vinod Sharma points out in this blog post, women can feel hesitant and less comfortable when speaking with enumerators who are men. Hence, a sufficient focus on hiring, supporting, and training women enumerators has been essential for us.

How can we improve data quality in surveys with women? 

A few steps enumerators and project teams can take to work towards data quality improvements effectively are listed below. 

Emphasis on the gender lens during enumerator training: During the hiring process, the team should ensure sufficient representation of women enumerators (tips are outlined in Vinod’s blog post). Enumerator training is a central space for underscoring the importance of gender-sensitive practices. Surveyors need to be aware of the different settings, challenges, and behaviors linked to respondents’ gender and crosscutting caveats, such as social class or caste. Training should include guidance on building rapport with the respondent, particularly in cases where they are initially uncomfortable. Practice sessions could help enumerators build soft skills and create a comfortable environment for the respondent. Sensitization of enumerators on the importance of respondents’ ease is not only essential from the data quality lens but also from a dignity perspective. In cases of sensitive or women-specific research topics, the respondent mapping should be done in consideration of their and the enumerators’ gender. 

Importance of the survey setting: The setting of the survey can impact respondents’ comfort levels, ability to truthfully answer questions, and confidence levels. Enumerators should try to conduct the survey in a separate room or location from where other household members, community members, or neighbors are present. In cases where the participant expresses a group setting preference, the enumerator should ascertain that the survey is only being conducted with the selected individual. If a partner/husband is dominating over the women present, the surveyor should politely request them not to interrupt or move to a different location for the survey duration. The balance must be carefully maintained, as surveyors are in the respondents’ households and would not want to imply disrespect.

Employing a gender lens when analyzing the data: In cleaning and analyzing data, it is easy to forget that each data point is a response of a human being in a specific context. Whenever possible, we should conduct gender-disaggregated analyses and investigate differences between gender subgroups. Looking through the literature can help us contextualize our findings and understand what other research in the field states from a gender lens. It is good to ask ourselves why we see a certain difference – would we expect women to have a different opinion than men? Can this be linked to the question framing or the performance of a specific enumerator? Could the data collected from women be more polluted? Or are there other reasons interfering in between?

Why do we need to care? 

Data quality defines our recommendations and their impact – inaccurate capturing of women’s voices perpetrates systematic inequalities. No experience is gender-neutral, and surveys are no exception. A baby crying from another room, the list of household chores occupying women’s minds, or a husband interrupting the responses are only a few observable, surface challenges that may pollute data collected from women. Data quality is essential – accurate data helps us bridge systematic gaps, among others, those linked to unequal healthcare access, safety, or media presence. Data needs to be collected in a gender-sensitive manner to minimize the coloring under the lens of systematic subjugation structures like internalized patriarchy. Awareness and training are essential to bridging the data quality gaps between men’s and women’s responses. Until we close these gaps, our efforts will not be as efficient as strong as our intention to achieve equal gender parity.

Acknowledgment

I would like to thank my colleagues at IDinsight for their comments and reviews – thank you, Abhishek Sharma, Priavi Joshi, Vinod Kumar Sharma, and Puneet Kaur!