Understanding Impact Measurement – A Step-By-Step Approach for Investors

Collecting meaningful data is a long process. It requires a well-thought-out framework, a dedicated team, and the right tools to support staff in their day-to-day activities.

It also involves understanding attribution and how outputs and outcomes indicate (rather than the cause of) short and long-term changes. Moreover, it’s about collecting qualitative data and telling stories.

1. Identify Key Stakeholders

Getting stakeholders involved is vital in understanding how measuring impact investing will be received. If the people most affected by an effort feel they need to be included or understand how it will benefit them, they’re less likely to support it. 

It can be a challenge, especially for nonprofits that need more tools (software to track data, staff, or time to collect and sift through it) or funds needed for such an effort.

Stakeholder mapping helps identify those who have power or influence over a project and can help you map their relationships with others. These individuals or organizations are called primary stakeholders, and they can range from those who stand to gain from an initiative to those who may be harmed by it. 

For example, a program to reduce domestic violence might result in fewer calls to emergency rooms for victims, but it could also increase the number of cases taken by police.

It’s essential to understand what types of attribution are available for each metric you choose. For example, a food bank can measure success based on how many people it serves weekly or how much food is donated to its establishment. But knowing how those outcomes compare to broader community goals is equally important.

2. Identify Key Performance Indicators (KPIs)

Identifying the right KPIs (key performance indicators) is essential in understanding impact measurement. KPIs are quantifiable measurements that track progress toward a company goal, such as increasing sales or improving customer satisfaction.

When choosing your KPIs, it’s essential to consider how attainable the goals are and how they relate to your overall business objectives. Then, decide how frequently you will monitor and re-examine the data to understand performance over time. Finally, determine the best format for communicating your KPIs to stakeholders and create a clear timeline of when you will share results.

Many investment firms gather KPIs by asking portfolio companies to report on the same data points they would otherwise collect as standard business operational metrics. This approach helps prevent overwhelming portfolio companies with additional data requests. It allows them to focus on their core business operations while providing investors critical insights into social impact. 

However, more than high-level data points are needed to demonstrate a company’s true impact. It is due to the challenge of causal attribution, in which it can be hard to determine whether community-level changes result from a particular program.

3. Collect Data

An impact evaluation requires the ability to determine what would have happened in the absence of the program. It is known as the counterfactual and is at the heart of any good impact study. Many impact studies need to be revised in this critical area because they attempt to answer the wrong questions or methods.

Choosing a suitable method of data collection is also essential. Organizations must decide who will be surveyed and how to conduct the survey. They should consider whether to use a standard set of metrics (such as IRIS+) to compare with other impact investments or create indicators to reflect the needs and resources of their project.

They must also decide what to do with data once it has been collected. For example, some organizations will share anecdotes highlighting success stories to encourage action or decision-making. 

Anecdotes are a valuable tool but should not replace data-based evidence. This step can also include selecting the best way to communicate results, including storytelling and other formats that make impact information more relatable.

4. Analyze Data

The push to demonstrate impact has led to many organizations collecting more data than they have the resources to analyze. It can distract from monitoring and evaluating performance and compromise efforts to collect the correct data. The problem is that many impact metrics are difficult to understand and apply and can be skewed by external influences.

Choosing the right metric for your situation is essential. For example, a social impact organization may want to track the number of people downloading their information and education materials (this is their activity or the ‘X’). 

They will also want to monitor the outcomes of this work, such as the change in diets of the population or the amount of fruit and vegetables eaten by residents.

An appropriate metric is the best way to measure these outcomes and impacts. It could be a ‘difference in differences’ approach, which compares two groups pre-trial and post-trial to see how one has improved over the other. This method is particularly effective for measuring ‘individual level’ impacts, such as improving literacy rates.

5. Communicate Results

When a for-purpose organization collects and analyses impact data, it can help them substantiate their work and show stakeholders they are essential in creating positive change. However, collecting and analyzing this data is just the start. The next step is communicating the analysis results effectively to ensure stakeholders understand the information.

It includes sharing quantitative (data that can be measured numerically) and qualitative data, less tangible but just as meaningful. Qualitative data can be provided through anecdotes, participant stories, and case studies.

Conclusion

In the case of a food bank, for example, it would be helpful to highlight their progress against the broader societal outcomes that they seek to achieve. It may be difficult for a charity to claim full credit for a result, such as a reduction in hunger, given the COVID-19 pandemic and other factors they could not control or influence.