What does it take to be an AI-native organisation?

Lorreen: Welcome to the IDinsight podcast! It’s so great to have all of you join us today. We are excited to have this conversation, to hear from you. This is a very, very interesting topic. And we’ve been preparing to hear from all three of you. So, as we’re starting off, I’d like each of you to introduce yourselves. Tell us what your role at IDinsight is and how you’ve been a part of IDinsight’s internal AI transformation.

Rob: Hey, everyone! I am Rob. I am Director of Impact and Transformation. I lead IDinsight’s internal measurement process. We are trying to learn from our impact and get better. I lead our internal AI initiative, so trying to help IDinsight deploy this tool to generate more impact. And, I also support external partners with their AI transformation, so we do some advisory work as well.

Linh: So I’m Linh. I am a Senior Associate at IDinsight. I’m based in Hanoi, Vietnam. I’ve been at IDinsight for like over four years, but since last year, around August, that I have been on Rob’s Impact and Transformation team, for particularly the AI initiative. Yeah, I’m one of the people working directly with teams to kind of close the gap between the AI strategy and initiative we’re having with the tools and the changes that people need to make in their day-to-day work as part of it. And, I also support the external services that we are offering as well. Okay. Fola, on to you.

Fola: Hello everyone. My name is Fola Salami and I’m the Information Security and Systems Lead at IDinsight, which basically means that I spend a lot of time thinking about AI governance and the management of product and technology risks at IDinsight. And I’m excited to, you know, offer some of that perspective to this conversation today.

Lorreen: Thanks, Rob, Linh, and Fola. So, today we’re going to focus on the practical side of what we are calling internal AI transformation. What do organizations need to do to adopt AI internally, and how can we how can we give as practical advice as possible? We’ve been doing this for years now, and we are so lucky to have the three of you who’ve been leading this initiative within IDinsight to talk us through some of the decisions, the trade offs, the considerations of safety risks, etc. that you’ve had to think about as you’ve been implementing this within the organization. So, before we even get into the “how” we want to talk about what is being AI native, like what does that even mean? Tell us about what success looks like from an internal AI transformation perspective. What are we trying to achieve? What kind of changes do we want to see as projects get delivered?

Rob: Yeah, I can get us moving. At the high level, what we’re trying to achieve is we’re trying to deliver more on our mission, right? Like we are a social impact oriented organization, and we see AI as an opportunity to further that mission. Like all of the work that we do in AI is centered around that mission. As we work with partners, we help them, you know, maintain that focus. There’s a lot of different conversations about what AI is, what it can do, etc. It can maybe transform your business model. Maybe it can help you increase the efficiency of the services, or the value that you provide to others. And that’s true for IDinsight. If we can become more efficient, if we can generate new value streams, all of that is trying to impact more lives, trying to be more cost effective for our partners, provide more value for our partners, etc. And, it’s just a tool. Oftentimes it’s the wrong tool, sometimes it’s the right tool. I think, one of the things that we want to do is sort of be balanced in our view of this. Some of us are skeptics. Linh would call herself a skeptic. Some of us are more excited about it. But we want to be very clear that AI is not the strategy; it’s a tool to achieve the strategy. When we think about AI within IDinsight, with our partners, we’re really trying to figure out like, where does AI link in clearly to your strategy to create more social impact. Where does it fit with the problems you’re trying to solve and where it doesn’t it. There’s plenty of things that AI can not solve for you. It’s not going build trust with clients. It’s not going to solve your cultural issues. There’s a lot of things that AI isn’t good for. It’s about figuring out, within our strategy, within what we’re trying to achieve, where is AI an opportunity for us? And we do a lot of writing. We do a lot of coding. Where, you know, large language models are language models. And so they have a lot of strengths towards those activities, and we’re trying to deploy them within those activities to advance our strategy, advance our impact. But there’s also a lot we’re trying that we don’t think AI is like a one size fits all solution for.

Lorreen: Yeah. I want to hear from Linh, who is a skeptic. What is your perspective of what success looks like?

Linh: So for me personally and I also mentioned this before when I told my team why I wanted to join the team, despite being like a skeptic is because I see, I know for a fact that it’s going to be very beneficial for IDinsight if it’s done right. So yeah, well, like I seem, I know that I can contribute to you like creating an enabling environment. So like our colleagues can use AI safely, effectively. They don’t need to understand what a large language model is. Some of them do. But a lot of us don’t. So if we get like the tools and policies and guidance right, and allow people to experiment and use AI where they know it will work for them best. So that we can dedicate more time and energy to a task that AI cannot do for us, like building trust with clients or like running surveys in rural Philippines. That’s kind of places. Then, yeah, I think that would be what success would look like in my opinion.

Lorreen: Thank you. Yeah. I want to hear more from Fola actually since Ling you’ve mentioned this whole like getting it right. So Fola you take care of our systems and security and just what is getting it right look like for you from that perspective. And like do you have any specific things that you think orgs need to have in place before they even begin to pursue like this ‘AI nativity utopia’ that we are talking about?

Fola: Absolutely. Thank you so much for that question, Lorreen. I think I’m fully aligned with Rob and Linh in terms of what are those outcomes that we are looking for. The only other thing that I’ll add in terms of an extra dimension when we think about outcomes, is making sure that people can actually do that in a safe and compliant way. So thinking about how does AI fit into our risk environments? Are there risks that might be introduced by deployment of AI, for example. Are there new questions that we might need to ask about mitigating those risks? I think governance is what I believe is the first step in sort of like any AI transformation. And, I am full aware that it doesn’t always happen that way in the real world, right? We’re all excited about AI. We need to move fast. We need to take advantage of AI. But ideally, governance should come first. And when we think about governance, especially AI governance, to certain people it might sound like something strange. But to, you know, debunk that, like AI governance is, made up of things that we are really familiar with, things that we are already dealing with. So it’s a component of enterprise risk management, which every organization already thinks about. It’s about how can we optimize the risk environments to make sure that, one, we’re not being too conservative so that we miss out on opportunities. But when we’re experimenting as well, we’re doing it in a really responsible manner. So what might that look like in terms of, like, tangibles? Right. Policy is a great place to start for an organization, thinking about some form of cross-functional, like governance but within your organization as well. That can really be responsible for steering things along. Like flagging where issues might be and, the places in which in the areas in which we are applying AI, for example. You’ll find out the like at least what I’ve seen, from where I sit within the organization is people get very excited about AI. We start out with policy and we get very excited with AI. Or like the governance function within the organization really exists to be able to say, “Oh, in all of this excitement let’s not loose our way in terms of what we’ve defined in policy, in terms of how we can do this right.” And so really the first place to start is your policy. It’s about your structures of accountability as well. And in many organizations, what that looks like is a cross functional body that is responsible for that. I don’t think it should be just the Information Security Lead. So there are all perspectives that like, you know, represented within the organization. It should be a bunch of cross-functional leaders and even all those within the organization as well. So, you know, like everybody’s interest is reflected. And so that is sort of like the outcome from my own side of things.

Lorreen: I actually want to talk about this a little bit more, just maybe for two minutes from a practical standpoint. So we’re talking about social sector organizations that are often resource constrained, that probably can’t afford to have a whole team dedicated to information systems and security. So from a practical standpoint, how many people are we looking at? Is it like two people? Three people? Five people? Are there specific tools that people should think about and consider that would, you know, form part of this governance framework? How long is a policy, like 15 pages? How rigorous should people be when it comes to policy given just thinking about how quickly this AI world is changing and how excited, like you said, people are to adopt this tool.

Fola: Amazing. Again, Lorreen, I think I’ll touch on something that Rob mentioned. There is no one size fits all for every organisation. Every organization is different. We all have various strategic goals. For example, IDinsight is an impact driven organization. So everything that we do needs to connect back to that impact. So that’s our north star as an organization. So we’re looking at it from that lens as well. Also may be connected to impact, and the context of IDinsight is ethics, which we take very seriously. We have an internal ethics framework that we abide by in terms of the way that we work as an organization. That might not be a feature in many private sector organizations or even slightly different organizations in the sector. So it’s not a one size fits all. But when we go into the specifics, for example, I think there are some tips that I can share that I think could be useful across. On the people side, I don’t think that more is necessarily better. Because again, it depends on, on the organization. I have seen organizations where that’s sort of like, you know, AI governance function is maybe even like a part-time capability for people that I would do jobs within the organization. Again, it’s about the scale of operations. I’ve seen organizations where it’s one person, I’ve seen organizations where it’s multiple people. So again, every organization needs to answer this question for themselves to see, like what is appropriate for the scale of operations and what is appropriate for our ambitions as well in terms of our use of AI, for example. So like a foundational model provider like OpenAI, Anthropic they probably have a much larger team thinking about these things than organizations like IDinsight that deploy AI. Actually, I should mention the like we also, you know, actually do some work in terms of like building models as well. So we’re definitely not the last on the chain in terms of like, thinking about, you know, the most appropriate safeguards in terms of people. So that’s sort of like the people piece. On the policy, I think one very useful tip that I’ll share is that I’ve seen a lot of organizations actually write one policy in terms of like a document that is meant to guide how people think about responsible AI use within an organization. And I actually I don’t think that’s the right approach. Yeah. I’m happy to be corrected with regards to this. One, I feel like, policy is very slow moving within many organizations, as we all know. Many organizations barely review their policies, even once a year. AI is sort of like the fastest moving thing in the world today. So, you know, like, it’s impossible for your policy to keep up with sort of like all the changes with AI. And so what we’ve done at IDinsight that I think other organizations can imitate and see if they can learn from that is we have a separate policy, which is our high level principles about how are we going to engage you to as an organization. Then we’ve got like a set of guidelines that are targeted at how people should use AI. Even cascading that down, even further, Rob and Rob’s team, I think they have done a lot of work in actually even breaking that down further to actual point in time infographics, for example. that we share with teams around, like do’s and don’ts. I think those are all very useful in thinking about sort of like the, the guidelines. So we’ve talked about people, we’ve talked about policy. Finally I’m going to talk about tools. Right. Actually, rather than tools. Let me talk about some other steps that might be necessary before you think of deploying AI. Well, something that is top of mind is, how do you prepare your organization for AI, right? Like, how do you prepare your data? And we know that data is at the heart of any important AI transformation. The real value in AI transformation is being able to, like, leverage your data that is unique to your organization to be able to generate really useful outcomes. And so one of the things that you want to do as an organization is do an audit. Like what do we have as an organization. Like what is this information assets that we have within our systems. How do we categorize them? So and there’s a process in I.T that is called data classification. That actually enables us to do that. Data classification is important in two very important ways. In terms of our systems, for example, when we do some form of like automatic, you know, classification of data, based on sensitivity level, then I automated like, you know, we call this class of tools not to give you confused data loss protection tools. They are able to be more effective, right, when something that is highly sensitive is about to leave the organization. Tables like maybe like, you know, block that action or alert somebody, you know, to be able to to do something about that. So yeah, that’s something that I think is very important in terms of other tools, the AI firewalls these days that I’m really excited about. We’ve not deployed that personally. at IDinsight, but I think is something that, I’m excited about in just the ways that they work. Essentially, you’re not, thinking about, you know, risks from an individual tool perspective. You’re thinking you’re looking at it from an organization, systems, environment perspective. And so I think that’s a very important way to solve the problem. So that’s also another important thing. Right. And say what? Again, I feel like I’ve spoken a lot and I’m maybe getting into the weeds of the technicalities for some of our audience. But I think that paints a really good picture of some of the things that need to happen before you rule out AI. Maybe Rob, you would like to contribute to that?

Rob: Would love to just jump in on one, you know, to your point earlier about the connection between governance and the higher level objective. Governance is a foundational, it is really important to start at or have in place at the start because of its link with the overall mission, just as well as saying, you know, as a social organization where our goal is impact, like governance is going to, for one, help us avoid the downside risk, right? Like we are trying to use AI to provide more value and to generate, you know, more impact. But we also need to be really cautious, especially in the context that we work in about, like not creating harm or doing things worse. Right? Like we handle sensitive data, we handle human subjects research or helping our partners make sensitive judgments about populations that they’re working with. And we really want to make sure that not only but can we use this tool to help us get better, but that it’s not making us, you know, make worse decisions or behave unethically. And so guarding us at downside risk is very much part of this overall impact journey. And two, it’s extremely important for adoption. Our teammates, don’t want to mess up. They don’t want to do things wrong. They don’t want to use AI in a way that’s going to, you know, create these harms. And if they’re not sure how to act safely, they’re not going to act right. Like the the behavior that that engender is, is just people are like, okay, I’m not really sure what I’m doing here. I’m not sure what the rules are or how to do this safely. And so they’re just not going to act. But we need people to act. Part of this adoption journey, this change management journey, is people need to try it. They need to experiment. They need to test it. They need to, you know, pilot things in their work. But if they’re not sure how to do it safely, they’re not going to do anything. And so having that policy and that governance foundation in place is extremely important for unlocking that adoption and that impact journey.

Lorreen: That’s a really helpful perspective, Rob. So let’s let’s talk about this also from a leadership perspective. So you are leading this organization. Let’s imagine you have strategies in place. Your governance is in place. Your guardrails are in place. You are ready to go. There are like a bajillion AI tools out there. And everybody who markets their tool says it’s the most important, the most legit tool you should implement in your business organization today. How do you make the decision about where to begin your investment? Again, talking to social sector orgs that have probably a limited budget, what investments are important, what looks like it’s important, but actually isn’t that people shouldn’t consider at the beginning.

Rob: I can get it started by sharing mistake that we made, which was. So we started really looking into AI for internal work back in like late 2022 to early 2023. And for whatever reason, we got really we were really focused on like vendor built tool. So rather than working with enterprise tools, you know, Gemini, ChatGPT, Claude, we were looking for, like task specific tools that, like, help us write content notes or help us do qualitative data analysis. And I really think that was not the right place to start. And it’s not where we spend most of our energy. So rather than trying to find these customized tools for these very specific solutions, most of the work that we do internally at IDinsight and with partners is customizing enterprise tools to our specific context and doing that customization ourselves. So, we don’t have 60 different, vendor built tools across the enterprise. We have a few of those for very specific situations where like it’s really justified, but a lot of our work is just taking those enterprise tools and customizing them. And so what I would say is get to an enterprise tool as quickly as you can. There’s a lot of other, data security and benefits from having an enterprise tool and spend a lot of time trying to figure out how to customize that enterprise tool to your specific problems that you’re trying to solve.

Lorreen: Okay. That’s that’s very insightful. That’s that’s really, really incredible point of view. It just really makes me feel more calm about all the YouTube ads I’ve seen of like, every tool for every single thing. But how do I even keep up and which ones do I adopt, which ones do I ignore? I’m gonna bring Linh in. You said you’re more in the execution of the AI transformation strategy. And I’m just curious. Rob talked about, you know, guardrails giving people the confidence to be able to do something. But I also know there’s a tension between committing time to learn how to use these AI tools versus, you know, just like not committing time and people sticking to how they’ve always done things in the past, even if their old way is incredibly time consuming. So what are some of the tactics you put in place to encourage teammates to adopt AI into their day-to-day workflows?

Linh: Yeah, so honestly, like the time it’s like investment is a hard question that like, I think our team is still wrestling with because I’m sure you know yourself, but we have a bazillion different responsibilities on a day-to-day basis, and asking people to spend even an hour or two per week to learn something new is hard. I think a couple things that, in my opinion, have worked out more or less well for us so far is that the the first thing we we have to make AI not scary. Because tech can be a scary thing. A lot of people still come to me, and still say that I’m not the technophobe. I don’t know how to do this well. I am scared that I will mess it up. People are scared. Like they you could put in a wrong prompt and their whole drive folder is gone. And, firstly, we have to reassure them that that is not possible. We have version control. Everything’s in the trash. You can restore them. It is fine. Fola is here for that reason. And we’ve set our system, we configure them in a way that is like as fit as possible for people to use. They’re not allowed to access or have rights to do things that they’re not supposed to. And we communicate that to people. Right. And then also we need to talk about all this, like AI futures, AI tools, like AI custom built tools that we have in the most accessible language so that people know that is something that, like, they can get a handle on easily. So, yeah, basically, I just try to meet them at the right level. One way, I like to think about this personally is like I am a coffee enthusiast. Like I’m from Vietnam. I take my coffee seriously. At the same time, I know that people have different taste in coffee, and, so the best coffee is a coffee you like. So its the same way like the AI that works for you is the AI that you understand and you’re comfortable using. So obviously, our job here is to try to help people to be more fluent, so to speak, to be more comfortable with a more diverse range of AI features and capabilities. But at the same time, we want to make sure that where people are at, we can meet them there. We have the right resources, the time, the responsiveness. So they know that someone is there to support them if something happens.

Lorreen: I want to stay on this like human side of AI adoption for a little longer. Fola, you’ve implemented a lot of tech-level transformation or change that has happened across the organization, whether that’s adopting new tools for security reasons or, you know, password managers across the organization, etc. And, obviously, Rob, you’ve led the AI transformation. Do you have a recommended approach? Should this be top-down? I mean, I’m sure you can make it safe and you can, you know, create how-to videos. You can do all you can from a bottom-up perspective. But you know, people might still be resistant. And I’m curious if a top-down approach is also necessary or appropriate, or does it make the transformation easier or harder? How should leaders think about just getting the whole organization

Rob: It’s a great topic. You know, some organizations are totally bottoms-up. Like how can we, you know, enable people at the front line to solve problems and, you know, move the organizational trajectory faster? Some organizations are extremely top-down. Maybe you heard the story about the CEO of Coinbase. He asked all of his engineers to get on AI on a Monday. And on Friday, he checked who had locked in and who hadn’t, invited all those people to a meeting and those that didn’t have a reasonable excuse. He fired them all. That’s like extremely top down. And we’re trying to figure out what that balance is. At IDinisght, we’ve kind of landed on a mix mostly bottoms-up, but with a team or group trying to build the enabling environment that is a little bit top-down. And we’ve talked about pieces of that. That’s getting an enterprise tool in place, that’s getting your policy in place, that’s getting your governance in place. And I think just trying to to help set up organizations or teams across the organization to accelerate their learning journey. So it’s about, making learning bite size so that they can do it within their busy schedules. It’s about setting up learning avenues or forums where people can share what they’re learning, how they’re using it. We’ve had like giant chats across across regions. Another thing that we’ve spent a lot of time doing is kind of workflow audits within teams. So we have worked, Lorreen, with your team to have a little session to say, like, let’s do some diagnosis where where are your AI opportunities, where the problems you have interact with AI superpowers. We did it with Fola’s team as well. So trying to sort of like build the strength across the organization. We do think that you know, in the long run this can’t be owned by a central team, alone. Just three people are not going to have the bandwidth. They’re not going to have the understanding of the work to identify all the organization’s biggest problems and solve all the problems. It really needs to be that everybody in the organization is like an AI augmented expert at that job, right? Like I should be the AI augmented comms expert. I shouldn’t be the AI augmented. systems expert. Those are other people. And so we’ve got to figure out a way to get the entire organization to be, you know, that within their role, within their their expertise. So we’ve landed on this kind of enabling environment balance.

Fola: Great question, Lorreen. I think I’m totally aligned with a lot of the things that Rob has shared as well. In terms of like really breaking down that resistance to change. I think one of the components, especially when we look from the lens of change management frameworks is how do we stir up the desire in people to actually like, want this change that is coming? And I think for AI on like a lot of the other things that I’ve implemented, I think it’s an easier sell, in saying you’re going to have more time for you to be able to focus on more strategic activities that connect to your job. And like, especially in an organization like IDinsight, connecting that to the impact that like we bring about as an organization as well, you know. The better, the more efficiently we can work as an organization, the more people we’re able to reach. And so really thinking carefully about what’s the carrot for everybody in their roles. And I think those workshops that I talked about, I think do a good job of like, you know, helping us to identify that carrot. And, I believe that the central components of the external facing work that we do as well in terms of AI transformation.

Lorreen: Thank you both Fola and Rob. So we talked about how we did this as IDinsight. We’ve talked about how we started with governance and policy and guardrails and then we’ve talked about the tools and how we, you know, tested different approaches for which tools would be appropriate for which these types of work. And then we talked about change management. How do you inspire or, you know, get the whole organization to to be excited about this transformation. And we’re continuing to learn on that journey. But we’ve done this for a few years, and now we are starting to support other social sector organizations in, in the development space to do this themselves internally. And I just, I want to hear, from Linh like, what is the service that we are providing?

Linh: We basically offer to our peer organisations the kind of the same services that we are doing internally at IDinsight. We would like to support them at any stage on there AI change management journey. So something like advising them on the kind of guidelines, guardrails, policies, the kind of tools that would be best for them, how they can start figuring out, yeah, basically just starting the ground work from getting people and organizations excited. How they can start learning. Well, so that’s what we are a lot of way of doing it at IDinsight for our external partners. I feel like Rob would have a lot more to say on this than me, because it’s a lot more active in our client development.

Rob: IDinsight has been working in the AI space with our partners since 2018. We really started by building products to help solve very specific problems. Back then, it was machine learning, and we were trying to solve prediction problems. And over the last eight, ten years, we’ve really branched out to like a full suite of services around AI, and we continue to build products and solve problems for our partners. The work that Linh and I are most often doing is really on the internal transformation side. So trying to figure out how do you optimize your workflows, how do you identify AI opportunities, how do you build the right tools, how do you help your entire organization become those AI augmented experts that we were talking about? So helping them identify, do you have opportunities to build bespoke tools, custom tools that are maybe more facing with the populations that you’re working with. So we built tools like, that help connect, pregnant, mothers with health care information that help community health workers do better diagnoses that, you know, are really at the more programmatic level. And so our team can also help, you know, scope and identify those opportunities, build those opportunities. And finally, we also do a lot of work in the evaluation space. You know, IDinsight grew up in the evaluation space, and we’re really adopting that toolkit and methodology to evaluations. There’s some unique needs that AI presents that aren’t just about how the tool is working or what impact it’s having, which are kind of more classical evaluation questions. But like, is the model operating in the way we expect? Is it producing reliable and safe results, like how are our users interacting with it? So we’re also working in this evaluation space and helping people figure out how are these AI tools I’m developing working? Are they working as I expected, recording the impact I’m expecting. Are they are they behaving safely? Can I trust them to interact with the decisions that we’re making, which are often in these sensitive contexts? So it’s really kind of branched out to a full suite of services in this AI space. Some of them, Linh and I really get into the details in and some of them, it’s our engineering team. It’s our evaluation team. It’s different folks with different expertise across the organization.

Lorreen: Really, really exciting. So I want to ask a question about the evaluation piece and whether you’re doing this within IDinsight or you’re doing this with partners who we are supporting with their own AI internal transformation journey, how do we know it’s working and what do we measure? Like how can we tell that investing in all, investing in everything we need to invest in, to be able to make this possible for organizations is actually having delivering on the impact that is promising.

Rob: It’s really, really hard. It’s the same toolkit, but it’s a different type of problem. It’s a different type of tool that you’re trying to measure. So, as I talked about, there’s you might want to be testing the model. You want to test how it interact with users. But I’ll just say, like measurement is a really, really important part of this journey because of its value and learning and trying to get it right. And as an example, we one of the very early tools that we rolled out across the organization was to help with performance reviews. So, I’ll admit it, everybody hates writing performance reviews. It was like really easy to jump in on this because it’s a huge, you know, it’s like it’s just people have an emotional drag. And so you want to walk in through open doors and you want to find, places that will people will get excited to save a little bit of time or get a little bit help. And we wrote out this tool that was trained on our performance review rubrics and customized to the way that we want to deliver feedback and trying to guard against bias. We were really excited about it, and people really liked it and were really happy about how it worked, at least at the initial reaction we got. But then we did some follow up data collection to kind of dig in to like what was your experience? How did you engage with this? What was your experience of somebody receiving? AI supported text, and there were a lot of, or there was a sub group of people that said, hey, this feedback was actually a lot less detailed. It was a lot less specific. It was a lot less actionable. And therefore, while I got kind of higher quality writing and the performance reviews that I received, I’m really less sure on how to act on them. And that’s the value performance reviews. It’s to help you grow. It’s to help you learn. It’s to help you, you know, get better. And that value was reduced. And had we not laid out some specific measurement to follow up and see how people were interacting with this tool, we would have caught that. And so we’re actually about the relative new performance review cycle. And so we’re like, okay, how can we fix this. We’re going to do a bunch of iterations and try to make it better. But measurement is really important to figuring out where where you’re getting the gains and where should you dig in and go further. But also like where is it having these unintended side effects, or where is it having consequences that you didn’t expect? And what you can learn, can you learn from that. So, strongly strongly, recommend having a measurement practice along with your internal external what you know, transformation anywhere that you’re deploying AI tools. And it’s really critical to do your impact.

Lorreen: So a quick follow up on the learning and the measurement piece, should you have like a blanket way of approaching this or just saying, okay, we’re going to check everything that we roll out? Or does the measurement approach vary depending on what tool you’re using? But like what the tools purposes for? What would you recommend?

Rob: Yes, both. We kind of have blanket organizational level statistics that we’re tracking to say, like how are people using AI? Like do they think it’s saving them time? How does that vary by team etc.. But we’re also we also strongly recommend, you know, workstream specific measurement approach. “Okay, you’re working on performance review tool, can that get better?” and a product specific measurement approach. Okay, you’re helping community health workers do diagnostics, Like is it working? Are they getting better? Is it more accurate. Like what are those populations. How what are those populations experiencing. So I think like on all sides yes, at a high level. Yes. At a workstream level, yes. At a product level, there’s a lot of different places that measurement should come into an effective strategy that are going to help you get better, faster.

Lorreen: Thanks, Rob. It’s interesting to hear how we’re rolling this out and supporting other orgs that are going through this similar journey and just using what we’ve learned to help people get to their destination faster. So for anyone who’s listening and really wants to reach out, wants to learn more, wants to talk to you and have a chat about how you went about it and get some advice or wants to hire IDinsight for their own internal AI transformation, what’s the best way to reach out? Who do they reach out to? How do they stop this conversation?

Rob: Yeah, reach out to any of us. We’re happy to engage. We’ve had a lot of really fun conversations with organizations across. We’ve talked to governments, foundations, nonprofits, research organizations, consultancies, everybody. is at kind of their own unique place. We learn a lot from those conversations. We’d love to hear from you. And get some stories and share insights.

Lorreen: All right. Last question. If you could go back and start over. Oh, what is one thing that you’ve learned that that you know now that if you knew then you would implement.

Rob: I would have gotten to enterprise tools faster. I think they’re super important for usability, for safety, for data security. I think we’re happy with where we are. But I think I just got there faster and I would have pushed harder or tried to troubleshoot more on like a Champions Network or Champions group. It’s something that a lot of organizations are getting a lot of benefit out of. And what’s been a little bit rocky at IDinsight, and I really believe in the value of peer learning, peer sharing, people are going to, the most credible messengers are the people closest to you. And I think leveraging more of a structure like this could have helped us earlier at IDinsight.

Linh: Yeah, that’s my first thought was kind of the same for a slightly different reason that we I would have liked for us to get an enterprise tool sooner. Partially because, yes, it’s safer, it i s easier to get everyone on board on the same platform. To also, it would have been beneficial for us to basically build all the custom tools within that enterprise tool sooner so people can benefit from it sooner. In that way we could have gotten a network of champion earlier.

Fola: I think similar to Rob and Linh, I think it is that one point, we should probably have, invested in, you know, one enterprise tool sooner. From my side of the galaxy, it also makes my work a lot easier. Like, I don’t have to wrap my head around, a risk assessment of gazillion tools. One thing I like to leave for audience is like, more isn’t always better. There’s there’s a lot of AI FOMO that is going on right now. Everyone’s coming up with their latest updates each model is overtaking the last and I think that’s going to keep happening. But the earlier you sort of like zone into one core enterprise tool, with of course, a reasonably solid model. I think the more you’re able to invest really building around that tool, like Linh and Rob said, like customizing it for unique use cases, embedding it into everyday workflows. My personal philosophy is like where we will see all the gains at the end of the day is going to be how embedded AI is within our organization. And it’s easier to do that for 1 to 2 and 3000 teams. And, and really just like, how do we, enable access to like, highly contextual, useful data within the organization as well? You know, connecting one AI tool to your sensitive data is a quite different scenario from connecting 3 or 4, right? You know, so, yeah, keep it simple. Zone in on what works. in terms of like one ecosystem. And I think you will probably see some of the best use from that approach.

Lorreen: Really, really solid advice. I think that’s our episode today. Thank you, Rob, Linh, Fola, for sharing so candidly and genuinely what your experiences have been driving internal transformation, internal AI transformation within IDinsight, to what the challenges were, what you’ve learned, and I’m hoping our listeners can take something from this, and implement this as they themselves rolling out internal AI transformation in their organizations. And if you need support from an experienced team that has done this themselves within, social sector organization, you know who to reach out to. Thanks, everyone.