Should your social impact organization use AI?

Over the last year, our Data Science and Engineering team at IDinsight has been creating AI tools like Ask-A-Question and Ask-A-Metric. Recently at a panel on AI for Social Impact, the leader of a digital health organization asked us whether they should invest in AI. Many organizations in the social sector are wondering the same thing. What should social impact organizations consider when it comes to AI? What are the costs and risks? And most importantly, what should AI actually be used for? Here are some thoughts on these questions.

Should you be using AI?

The short answer to whether to invest is “yes,” but that’s not quite the right question.

Think about when people first asked, “Should we invest in computers?” Looking back, the answer was obviously “yes.” But the better question was, “What should we use computers for?” The answer depended on your organization and the people you served.

AI is similar. To cut through the hype, start with your problems, not the technology. Think about what challenges you’re facing, and that will guide you to the right tools.

You also need to consider what’s actually possible with the current technology and your budget. AI covers many different technologies—from simple prediction tools that can help target services to those who most need them (predictive AI) to advanced systems that can create images, videos, or text from descriptions (generative AI). Even the spell checker in your document software is a form of AI. Understanding what AI can and can’t do well requires some familiarity with its capabilities and limitations.

What are the different ways to use AI?

1. Improve how your organization works internally

Start by thinking about your organization’s internal challenges. Is information scattered and hard to find? Is valuable data that could help with decision-making locked away in inaccessible or hard-to-manipulate databases? AI might help.

At IDinsight, we created an AI tool, HubGPT, to help our staff find and use our organization’s knowledge more easily. We also developed Ask-A-Metric, which allows staff to ask questions about our data in plain language and get answers quickly.

Hallucination (incorrect or misleading results) is a common concern with Generative AI and can lead to hesitation in full AI adoption. However, for internal applications, the universe of information the AI is pulling from would be limited to your knowledge management system or your internal data. So, there’s little to no scope for hallucination. Beyond that, since these tools are only used by your staff, the risks are lower than when the tool is used by a large population outside your organization. You can train your team on proper use, and if the tool doesn’t work well, you can stop using it without affecting your reputation with the communities you serve.

2. Use AI to enhance your existing services

The next level is adding AI to services you already provide successfully. Assuming you meet some prerequisites, there are (at least!) three ways AI can help make existing services better. If you have a program that’s been working well for years, AI might make it better – perhaps by making the user experience feel more human-like, automating repetitive tasks, or helping you more easily understand text responses users provide through feedback forms.

REACH Digital’s “MomConnect”—a chatbot for pregnant women and mothers in South Africa to make health inquiries for themselves or their babies—was successful in incorporating AI because the service already had millions of users on their WhatsApp platform (it had been working well for years). IDinsight helped REACH enhance their existing service through an AI-powered tool, Ask-a-Question. This allowed users to find information more intuitively by typing questions in natural language instead of navigating through complicated WhatsApp menus (making the experience feel more human-like). This chatbot was also able to respond directly to low-risk frequently asked questions rather than waiting for a human response (automate repetitive tasks). And when responses were poor, users could give open feedback about what the problem was, and our AI-facilitated dashboard could analyze the qualitative feedback directly and provide suggestions for improvement (help understand feedback forms).

Think about your users’ experience with your services. Are there pain points where AI could help? If you pursue this approach, you’ll also need to consider data protection, safety measures, and following regulations.

3. Create entirely new AI-powered services

The third option is creating completely new programs or services using AI. Examples might include a new app that helps people access benefits, a WhatsApp service that answers farmers’ questions, a tool that helps community health workers with diagnoses, or a digital tutor that helps with targeted instruction—teaching at the level where students actually are.

IDinsight is working with Indus Action to build an AI-powered tool to help people in India find out if they qualify for government welfare programs. The tool is designed to overcome two problems of funds going unused: 1) people don’t know about these programs, or 2) they find the application process too difficult and time-consuming to complete when they may not actually be eligible. 

Digital Green produced a chatbot, Farmer.Chat, which “is designed to deliver tailored assistance to hundreds of thousands of extension workers providing advice to tens of millions of small-scale farmers around the world.” IDinsight is not supporting the development of this tool, but we are working to conduct a rigorous impact evaluation of its effectiveness. 

These new services can have a huge impact when done right, but they require a deep understanding of your users’ needs, careful design of the user experience, effective communication strategies, and extensive testing with actual users. Like any technology solution, success depends on how well you understand the people who will use it. And even then, as with any intervention, whether it’s really having an impact should be rigorously evaluated.

How should you get started?

There are three approaches to implementing AI, depending on your organization’s resources, experience, and needs:

1. Adopt [existing solutions]

If you’re new to AI, unsure about its value, or don’t have technical staff, start by adopting existing AI tools. This means using ready-made AI services that someone else has created and maintains and simply paying a subscription fee. This is how we helped MomConnect implement Ask-A-Question, the AI-powered chatbot that answers mothers’ health questions.

This approach is:

• Cost-effective: You can try it with a small investment—just the subscription fee.
• Low-risk: If it doesn’t deliver the value you anticipated, you can easily stop using it.
• Quick to implement: You can start using AI almost immediately.
• Maintenance-free: The provider handles all the technical work.
• Continuously improving: You get new features without extra effort.

Adopting an existing service for your AI needs is the most cost-effective and least risky. The downside is that the service might not perfectly match your specific needs, and you’ll have limited influence over how it develops.

2. Adapt [existing solutions]

Another option is to take an existing AI tool (especially open-source ones) that meets most of your needs and adapt it to your specific requirements. This is how we built IDinsight’s internal knowledge management tool, HubGPT.

This approach works well if you can find a solution that covers about 80% of what you need. You install it on your own systems and make changes to fit your specific situation.

The challenge is that you’ll need technical staff who can modify and maintain the system, and extending the tool might be difficult if it wasn’t designed to be easily customized.

3. Build [your own solution]

The final option is to create your own AI solution from scratch. This makes sense if you have very specific needs and existing tools can’t meet them. For example, recently we started building an AI Diagnostic Assistant for Community Health Workers since none of the solutions available provided the accuracy that we were aiming for. Similarly, Adalat.ai built an AI stenographer for the Indian Court System — none of the existing products were a good fit for their use case.

Building your own solution ensures the final product aligns perfectly with your vision, but it’s challenging. You’ll need a strong technical team, which can be difficult to assemble and manage. The process often takes longer and costs more than expected due to unforeseen complications. Additionally, you’ll need to handle ongoing updates and fixes, which requires continuous commitment and resources.

What challenges should you be aware of?

AI can be expensive to operate

Traditional software has a high upfront cost but very low costs for each additional user. AI is different – costs increase with usage because you pay for each interaction. For example, with large language models (the technology behind chatbots), you pay a small fee each time a user asks a question. This is how it works with OpenAI, Anthropic and others. An alternative is to pay a flat fee to host your own open-source model, like Llama. However, given the computing power required to run, this can become very expensive.

Large models are like swiss-army knives; they are versatile and are great to build your proof-of-concept with. But as your solution matures and scales, you should explore creating and fine-tuning your own smaller model for specific tasks to manage cost. This does require a substantial up-front investment—likely engaging with external AI engineers—but it can save money in the long run. This is exactly what we did for the triaging service on Ask-a-Question.

Evaluating AI quality is difficult

AI systems, especially those that generate text, can sometimes produce incorrect or misleading information. Checking everything they produce for accuracy and completeness requires significant effort and resources. 

It’s important to develop a robust monitoring system around AI generated content, soliciting feedback from users, and reviewing this information frequently. Fortunately, AI can also support in synthesizing the feedback.

AI is easy to try but hard to scale

Creating a quick demonstration of an AI idea is relatively easy. A skilled developer might put together a basic version in weeks. However, building a reliable AI system that thousands of people can use safely, that provides accurate information, remains stable under heavy use, and operates cost-effectively takes months of careful work.

Before embarking on this investment, make sure the cost of implementation and maintenance is more than offset by the efficiency gains of using AI by modeling cost-benefit carefully.

Conclusion

As AI continues to evolve, social impact organizations should approach it with a focus on solving real problems. Instead of simply asking whether to invest in AI, leaders should consider how AI might enhance existing services, improve internal operations, or create new opportunities for impact.

The right approach—whether adopting existing tools, customizing solutions, or building from scratch—depends on your organization’s needs, technical capacity, and budget. While AI offers tremendous potential, it also comes with challenges in cost, scalability, and reliability. As with any other investment, one should evaluate the expected costs relative to the expected benefit. And try not to be overly optimistic about either.

Thoughtful implementation, continuous evaluation, and a commitment to responsible use will help organizations maximize AI’s benefits while managing its risks.