A new framework aims to help non-profit organizations identify the minimum specifications they need when setting up their monitoring systems.
IDinsight’s work with governments and NGO partners often involves designing or setting up performance monitoring systems. These are systems that help our clients regularly collect and analyze real-time data on their programs to figure out what’s working and where more resources are needed to achieve social impact. Performance monitoring systems are similar to what some organizations call “impact measurement” software tools, which help them regularly collect data on their work and share this data with staff and key stakeholders to help them make informed decisions. While these tools often do not measure the actual impact of an organizations’ work, which would require a counterfactual, they do provide important ways for organizations to track key outcomes amongst their target groups and understand whether their programs are being implemented effectively.
There are currently a broad suite of “off-the shelf” impact measurement software tools out there, and it can be difficult to figure out which tool is right for an organization’s needs. Typically, these tools have several capabilities: they help you collect data (i.e. through surveys or data import), analyze and visualize the data, and share back that data in the form of reports and dashboards. When selecting the right tool for your organization, one option is to buy an “off-the-shelf” software like Impact Atlas or SocialSuite. Other options are to build a custom system yourself or to bring on a technical vendor to build the system for you. If you choose to build the system yourself, this will likely involve integrating other tools like SurveyMonkey, Google Analytics, and Tableau. Bringing on a technical vendor will enable you to build a fully-customized system for your needs. In order to determine which type of solution best fits an organization’s needs, it is critical to first determine your priorities and how the tool will inform those.
Drawing on our experience helping clients set up systems to collect data and manage their impact, we developed a framework to help other organizations identify their needs, which will then inform the type of monitoring system they might require. Below are key questions you should ask before you begin exploring different solutions. The answers to these questions are the “minimum product specifications” you can use to rapidly test different tools and select the best solution for your organization.
The first step is to determine what data an organization needs, how it will collect data from stakeholders and the types of data it will need to collect. It’s valuable to specifically consider what kind of data staff need to understand whether they are achieving their outputs and outcomes. For example, an education non-profit might want to collect both quantitative and qualitative data from students and teachers, and possibly administrators and parents. Depending on a school’s context, it might be necessary to collect data in-person using mobile phones or tablets, or it may work to send an online survey through email. The questions below will help think through an organization’s needs around primary data collection:
In addition to primary data collection, another important consideration is the systems you use to track and store administrative data and how these will integrate with the Impact Measurement and Management solution. This involves thinking about how data from current systems — such as your Customer Relationship Management (CRM) system, grant management system, enrollment and application platforms, attendance trackers — will flow into the tool. Some questions to consider are:
The next step is to determine an organization’s needs for data analysis, including the work involved in preparing the data for analysis and the types of analytical techniques needed:
After determining an organization’s needs related to data analysis, it’s valuable to then identify what needs to be reported. This involves thinking about various audiences and which reporting outputs will best resonate with the people you need to present insights to:
Finally, it’s important that your team has the capabilities to use the tool on an ongoing basis:
You can use your answers to the above questions to formulate your “minimum specifications” before you begin exploring different tools. In our next blog post, we share some key considerations for determining whether buying an “off-the-shelf” solution or custom-building your own is the right approach for you.
10 October 2019
26 May 2021
5 December 2019
23 February 2021