How your organisation can use AI to improve efficiency

With the recent popularisation of Large Language Models (LLMs) like GPT-4, the market has started witnessing the creation of AI-powered data management tools, which allow users to “chat with their data”. While varying in their approach and complexity, examples include tools like CoHere for conversational data querying, ThoughtSpot for interactive visualisations and reports, and of course, ChatGPT. 

Inspired by these innovations, we set up and deployed our own in-house AI-powered GPT tool for internal knowledge management (or “KM-GPT”), which we built in partnership with DanswerAI.

In this blog post, we share our learnings and a basic framework for other organisations to help set up AI-powered knowledge management systems.

Organisational knowledge is scattered everywhere. It’s arduous and expensive to curate. 

Knowledge is power. The ability to capture, access, and leverage institutional wisdom can be the difference between a project’s success or failure. Yet, many organisations’ knowledge management (KM) systems – i.e., the collection, categorisation, and analysis of organisational information – are like navigating a labyrinth.

In the social sector, where resources are scarce, organisations frequently face financial barriers that prevent them from investing in the development of quality KM systems. 

Historic KM solutions; such as databases, intranets, and manual filing systems, have been the go-to for many years. These systems rely on multiple, sometimes disjointed platforms, complicating the search for information and leading to inefficiencies in connecting people with knowledge. This can reduce productivity, efficiency, problem-solving, and subsequently, impact.

…so, the next best thing? Bring knowledge to people, instead of the other way around. 

The introduction of AI has revolutionised the approach to internal knowledge management.¹ While an information organisation system is still needed, AI can efficiently search multiple diverse data storage platforms, quickly retrieving, summarising, and linking to relevant information.

IDinsight’s own KM-GPT: A unified solution for streamlined search

For the last five years, IDinsight has been supporting social sector partners to use AI to improve their work. This demand and opportunity has contributed to a 3X growth in the size of our Data Science, Engineering and Monitoring (DSEM) team since its inception. In addition, the team recently received support to further advance our AI work from Google.org as part of its company-wide commitment to achieve the SDGs with AI solutions.

Given the impact we’re seeing through our AI-focused client projects, we saw huge opportunities for it to help us internally become a more effective organisation and deliver higher value to our clients for less money.

This partly inspired the creation of “HubGPT”, coined after our intranet named the ‘Hub’. IDinsight’s new knowledge management AI-powered chatbot, it was built to improve the ease and speed at which employees retrieve information. It connects across all our organisation’s various knowledge repositories – namely; our public website, intranet, specific Slack channels, and shared Google Drive – and is accessible both via Slack and/or browser.

We invested in building HubGPT so we could save ourselves (and thus, our clients) money in the long run. If, eventually, ~50% of our employees use it and it saves each of them 10 minutes per use, we forecast it will save IDinsight ~$100,000 per year!

How we created HubGPT:

While this initiative was intended to be entirely in-house, leveraging the full extent of our technical capabilities (…and OpenAI APIs, of course), it was proving to be a high-cost and time-intensive effort. Instead, we decided to invest in a partnership with DanswerAI – a new team already making great strides in this space – and switched to deploying their open-source tool on our servers instead of continuing to build an in-house solution from scratch.

HubGPT in Action:

In the two months since HubGPT’s launch, we’ve observed an active² adoption rate of 20%, an average of 16 messages/user per month, and a total of 1055+ meaningful messages sent to the tool. While the total number of unique users so far is fewer than we’d hoped (likely influenced by the fact that the chatbot was launched at the cusp of the holiday season), the average engagement per user is ~15X higher than our initial estimates.

HubGPT’s value has been felt beyond just benefitting the bottom line. It’s helped improve employee satisfaction by making tasks easier and faster to complete, thus freeing up bandwidth to focus on more productive work.

“HubGPT is magic, and I hope you are all making full use of it.” ~ Director, Nairobi office

For example, a Senior Associate in the Delhi office spoke of their experience using HubGPT when entering a new phase of a long-term project. They said:

“HubGPT is incredible. It helped me quickly understand our project launch and budgeting processes, saving my manager/s and me tons of time!”

Further, the chatbot HubGPT is now the first respondent to questions about our recent ERP transition, leveraging tons of documentation produced by our finance team. This has enabled members of our finance team to stop actively monitoring messaging channels, and instead take on more productive work.

“Thank you for working on this, it has really made my and the team’s life easier!” ~ Director of Finance & Planning, Delhi office.

Our Learnings:

1. AI doesn’t replace the need for a good underlying dataset – in fact, its success depends on it

It’s easy to underestimate the amount of work that it takes to collect, construct, and maintain the dataset/knowledge base upon which a data science model is built. However, the more time we invested in cleaning and structuring our input data, the better HubGPT’s output became.

2. We may be able to go it alone, but doesn’t mean we should

When faced with the core organisational “make vs. buy” dilemma, we learned that, for a space that’s evolving as rapidly as AI, partnership was the most efficient way to go. While our data science and engineering team has the expertise to build this, our collaboration with DanswerAI helped us conserve time and resources while bypassing foreseeable obstacles.

3. Embrace (and fix) the bugs

It isn’t ‘cutting edge’ without ‘cutting teeth’. We learned that pioneering technology comes with its share of infancy troubles – bugs and glitches are par for the course. Instead of trying to prevent them, we’ve accepted their inevitability and have shifted focus towards iterative improvements. With every bug and glitch comes the scope for improved functionality, continuous refinement, new ideas, and product/feature enhancement.

4. Building the tool is 40% of the battle…change management is the other 60%

The key influencing factors to maximising HubGPT’s efficiency gains are how many employees use the tool, and how often they use it – and those numbers won’t increase by communicating the existence of a new tool, no matter how ‘cool’ it may be. Our colleagues are busy and inclined to stick to what they know during crunch-time. After realising that mere announcements won’t cut it, we moved from detailed guides to interactive demos, Q&A sessions, and direct feedback collection. We’re promoting HubGPT through KM responses and encouraging power users to share their stories in forums.

5. Creating a culture of innovation means actively trying to innovate

The DSEM team prioritises fortnightly hackathons dedicated to experimentation, innovation and the occasional knock-knock joke. This solution was prototyped on one such Friday evening, courtesy of this article, and it was the positive results of that 30-minute demo/testing that led to IDinsight investing in this initiative.

How can you build this for your organisation?

After going through our journey, we think other teams and organisations have four main ways to set up AI-powered search tools like HubGPT for internal business use –

1. Build a system:  This is best for large organisations with strong engineering teams and the potential for big gains if they build a system with the highest level of performance – imagine a system that can handle thousands of queries in seconds, has a 0.001% chance of giving the wrong answer, and no data goes to a third-party. A useful heuristic to think about is whether the cost of building and self-maintaining a custom system will be offset by the gains of the system in the medium to long term. This is also the most expensive path to building this system. For example, a business or industry where success is influenced by rapid and accurate knowledge management (like customer service/grievance redressal).
2. Host a system: This is best for organisations that have an established engineering team and are constrained by either cost or, given their business needs, they don’t need a bleeding-edge solution. Such teams can use free solutions and tools and mainly focus on deploying/setting up the system on their servers and handling maintenance and updates. Tools like Danswer (open source, non-cloud version), LlamaIndex, Langchain, etc. are great for such cases. This tends to be the cheapest option while also maintaining a reasonably high level of security and performance.
3. Outsource the system:  This can work for cases where the organisation does not have an established engineering team and does not need a bleeding-edge solution. In such cases, service providers that offer off-the-shelf end-to-end, cloud-hosted AI search solutions tend to be the best bet. The key compromise would be related to sharing data with a 3rd party and having a medium to high budget.
4. Build a customised system (with support from an organisation like IDinsight): If you don’t have an engineering team and want to work with us to identify a flexible solution that fits your needs. We can help you along a broad spectrum, from building a cost-effective and performant system to building the best possible solution that responds to your specific needs and context.

	Cost	Requires in-house data science skills?	Want the absolute best / good enough is not good enough	Data security
Build a system	Expensive	✅	✅	Highest
Host a system (what we did)	Cheap	✅	❌	High/Highest
Outsource the system	Moderate to expensive	❌	❌	Depends on the service provider
Build a customised system	Cheap to expensive depending on the requirements.	❌	Quality and cost tradeoff.	High/Highest

We hope that others will continue to share how they are using similar tools to improve their operational efficiency. Please get in touch or offer comments below about what you’re doing or approaches you’re interested in trying.