Good evidence as a starting point for more democratic technology

Countries and ministries are racing to write AI strategies. Companies are building models that will underpin services worldwide. As they do so, they hope their models will behave in alignment with a set of values. For instance, Anthropic has drafted a constitution for its AI models, covering everything from safety priorities to questions about the AI’s moral status. An in-house philosopher advises on the principles they should train AI products to observe. One review, as far back as 2019, already counted 84 ethics frameworks, and there are many more now. All of them reflect the moral views of their authors. What is a policymaker tasked with setting a country’s approach to governing AI to do? They cannot very well ask AI, which has been trained mostly by a particular group of people on the existing literature of views and values.

The social sector has swiftly established standards for preventing harm and measuring good practice at the level of individual AI-powered interventions. IDinsight has advocated for good impact and process evaluations, organised at four levels, from model outputs through to real-world development outcomes. Good research on user experience and impacts helps forestall unintended consequences each time, and helps close the model-practice gap, when technically sound AI tools produce starkly different outcomes depending on the user and context, to realize a technology’s potential. All of this is necessary, and must be taken up consistently across the sector, but it is not sufficient. Before any product is designed, the overall governing infrastructure is set; principles are laid down about what will be permitted for which populations, encoding values.  Here, there is a critical lack of process, and evidence. 

When AI’s benefits and costs are unevenly distributed, someone must decide what to prioritise for whom. Someone must make the hard decisions about who and how people will benefit or lose out from this economic and technological transformation. While the African Union’s Continental AI Strategy invokes a vision of AI and African values rooted in the idea of Ubuntu and collective community over individuality, what do citizens in different countries actually want that to mean for how AI treats them? Which partners are trustworthy guardians of citizens’ privacy rights? What types of AI work are good work? Should a Kenyan civil servant, sitting at the Ministry of Information, Communications and the Digital Economy, beneath a poster of the republic’s national values, recommend scarce public resources be spent on domestic data centres and more sovereignty, or spend on more basic enabling infrastructure, while relying on foreign cloud providers? As digital public infrastructure like the Huduma Namba national biometric ID system expands, should it be linked to AI-driven welfare targeting, given concerns about privacy and equity? If so, who decides what counts as a legitimate risk factor when algorithms sort citizens into categories of need? 

Matters of moral governance are in turn questions of democratic participation, since citizens have a unique right to weigh these moral questions. The Carnegie Endowment’s analysis of Africa’s AI governance landscape found significant variation in how much public input shaped national strategies. An in-depth analysis by CIPIT at Strathmore University concluded bluntly that “gaps remain in terms of genuine multistakeholderism, and genuine consideration of diverse views of users, especially vulnerable groups.” In Kenya, the constitution actually mandates participation as part of policymaking. Yet the strategies are being written, with minimal participation taking place. The citizens whose dignity is at stake are not yet part of the conversation. Democracies need feedback loops.

Values vary – not neatly, but measurably

One underexplored avenue for incorporating citizens’ voices is through good research. It is not underexplored because the questions are unanswerable. It is because the research that would answer them has not yet been demanded and funded, as AI accelerates forward. 

How people trade off their aspirations and fears around AI varies a lot around the world. When Moral Foundations Theory, one of the more comprehensive and empirical bases for moral trade-offs, was tested by Kenyan researchers, they found it consistently left out a pillar of morality centred on community. In my own research on dignity across cultures, I have found it helpful to think about where people locate their dignity. Much work has been done to correct for the ‘WEIRD’ distortions in our understanding of values. When researchers work together with people beyond rich countries, they report new and more relevant moral frameworks for policy decisions.

One of the most powerful approaches to LLM alignment has been the constitution developed by Anthropic. The careful, thoughtful team that have worked on that would benefit from a wider picture of what the world thinks. The data they have mainly relied on so far is a poll of American citizens. That already greatly changed their moral approach, compared to their own staff’s views. A new and remarkable investigation by Anthropic of 81,000 people is an important next step, but this surveyed existing users of the service. It would be far more just, and more powerful, if citizens around the world – starting with the least connected – had their say. New initiatives such as those by Collective Intelligence and Afrobarometer hold great promise.

How to measure people’s preferences

The sector knows how to do this kind of work. IDinsight’s research on measuring people’s preferences — partnering with GiveWell to thoroughly understand how people weigh health interventions against income-generating projects — is an example of upstream preference research that directly feeds into resource allocation decisions before any individual programme is designed. That work shows it is possible to systematically and rigorously gather citizens’ moral preferences, and to connect those preferences to the decisions that shape their lives. 

Studying people’s views on this is not easy; it requires careful methodological work. But it is necessary if we are to hear the voices of those who may be most affected.

Something equivalent is needed for AI governance. Not “does this chatbot have a good interface?”, or “how much did it improve recovery times?” but “what kind of AI health system do you want, and on what terms?” Not a one-off consultation, but systematic, ongoing input from the populations whose lives these systems will shape, feeding into frameworks before architectures are locked in. 

The stakes

We should be humble about what data alone can achieve. Gathering evidence about people’s moral preferences and experiences is essential, but it is only one part of the political solution and of wider democratic processes. Still, good research can make a valuable contribution by establishing that evidence base.

We can also together advocate for governments to pay attention to that data, once it is in place. The particular value of GiveWell’s work on measuring people’s preferences is not that it exists as a dataset, but that it directly plugs into their models and hence their budgetary allocations. They have an established system to respond to people’s evolving preferences. IDinsight’s research with women fish processors in Senegal showed that women in the artisanal sector directly and indirectly create hundreds of thousands of additional jobs, and they are often the primary breadwinners in their households. They identify lack of legal recognition, industrial overfishing, and lack of access to finance as key obstacles to their livelihoods. That evidence mattered, but it had impact because it was paired with political organizing and advocacy by groups like the Senegalese Network of Artisanal Fishing Women (REFEPAS) and Greenpeace Africa, who used the data to press the Senegalese government for legal protections that would help these women protect their livelihoods.

These decisions are being made now. Kenya, Nigeria, Ethiopia, and a dozen more countries have published their own frameworks. Major philanthropic funds on dignity are evolving their strategies. Companies are continually updating their models. These are often serious, forward-looking efforts. But the moral choices embedded within them are being made without the evidence that would ground them in the preferences and values of citizens.

AI technology contains within it the potential for enormous personalisation. When we understand how people trade off between values, AI systems can be built to respect that. But only if we ask first.

————————————————————————————————————————

IDinsight works to incorporate dignity into everything we do, including our strategic focus on AI. This post shows one way in which we think it is important to pursue dignity in technology. I am grateful to Mallika Sobti, Lorreen Ajiambo, Simran Saini, and Marc Shotland for their thoughtful reviews.