Impact. The daunting word so many in the nonprofit sector have come to dread. The hot topic at conferences, roundtables, meetings, and featured in countless blogs and articles. “Measuring impact” has become a mantra for nonprofit professionals around the world. Growth in the number of nonprofit organizations and faith in the good intentions of everyone involved seems to no longer be enough; evidence is now required.
Multiple approaches have been developed to help nonprofits define and measure their social impact, but a single, stand-alone, universally accepted framework does not exist yet.
Although measuring social impact has become increasingly important for nonprofits and their donors or funders, the task at hand is challenging – to say the least.
So, Why is Measuring Social Impact Challenging?
- It includes measuring complex concepts, such as “increase in self-esteem of an individual” or “increase in overall well-being”.
- Measuring social impact also means accounting for other factors that might have influenced the element you’re trying to measure i.e. to what extent can the change be attributed to the activities of your nonprofit.
- It is long-term and requires consistent and dedicated work throughout the span of several years.
- It requires a relevant system/framework and robust tools, which can be hard to develop, especially for smaller nonprofits.
- Even larger and more established nonprofits struggle to evaluate the quality of their impact measurement process.
- There are no set rules, so approaches and terminology can radically differ from one nonprofit to another, which makes communication and collaboration more difficult.
- The phrase “social impact” is increasingly vague and often misused to imply small changes.
Ok, I get that it’s hard. But what is even social impact?
University of Michigan defined social impact as “a significant, positive change that addresses a pressing social challenge. Having a social impact is the result of a deliberate set of activities.”
And what is the difference between impact and social impact?
“Impact” on its own implies an influence or effect on virtually anything or any kind of change from a current situation. “Social impact”, however, is grounded in the effect it has on a pressing social challenge.
Although what a “pressing social challenge” means might differ from one community to another, from one country to another, or even from one nonprofit professional to another, a minimum consensus exists on the fact that social impact should be more than a marginally positive change, and one that impacts a broader social issue.
Social issues include such issues as equality, livelihoods, health, nutrition, poverty, security, justice, conservation, energy use, waste, environmental health, resource depletion, and climate change.
Why is it Important to Measure Social Impact?
Albeit a difficult task, measuring social impact matters. This is especially the case for nonprofits since they seek to deliver their mission and bring value to their beneficiaries, and these are more difficult to track and measure.
Nonprofits need to develop the ability to know whether they’re making a difference and to know how to invest wisely so that they can do better over time. There are manyother reasons to measure social impact, and here are only a couple:
- By measuring impact, you gather data and stories that you can then use in your marketing and communications.
- You are able to get quantitative and qualitative inputs and feedback for your programs and activities, so you can improve them.
- By measuring impact, you are also able to report back to funders and be accountable.
- You can attract new donors. Project funding is increasingly dependent on the strength of impact assessment.
- You can share the information with other nonprofits and collaborate.
Where to start?
The most common approach to thinking about and measuring social impact is Theory of Change. Theory of Change looks at why and how can the desired change happen in a particular context. It sets out to connect what a nonprofit/program does (the activities) and its desired goals (the impact).
You can develop a Theory of Change for a project, a program, or an organization.
There is not a single, codified way to develop and activate a Theory of Change, but for the sake of this analysis we will represent it as a conceptual linkage amongst the following:
At the end of this process, you and your team should be able to answer the following questions:
- Who are you seeking to influence or benefit (target population)?
- What benefits are you seeking to achieve (results)?
- When will you achieve them (time period)?
- How will you and others make this happen (activities, strategies, resources, etc.)?
- Where and under what circumstances will you do your work (context)?
- Why do you believe your theory will bear out (assumptions)?
Theory of Change can be broken down into Inputs, Activities, Outputs, Outcomes, and Impact. Here’s how you can go about designing a Theory of Change for your nonprofit:
As an example, we will be using a nonprofit that gets young people into employment using support and mentoring.
Step 1: Inputs
Inputs refer to the supply of resources your nonprofit or project has. These may be financial, intellectual, human, premises or others.
E.g. Funding for places on training courses, or the time and training of mentors.
Do: Brainstorm all the resources you have at your disposal that are key to achieving your results.
Step 2: Activities
Activities are concrete actions of your nonprofit. These are primarily focused on creating improvements – changes – in the lives of beneficiaries. They refer to the work undertaken using inputs with the purpose of delivering the outcomes intended.
E.g. Group activity to develop peer group support or mentoring of individuals by employer-appointed mentors.
Do: Outline the key activities your nonprofit executes to achieve the intended impact.
Step 3: Outputs
Outputs are when your activities have points of contact with your beneficiaries. They are essentially results of the activities. Outputs are not about evaluating the effectiveness of those activities.
E.g. How many young people attend courses or how many start jobs with scheme employers.
Do: Think of all the relevant outputs/results that are important to measure when it comes to your desired impact.
Step 4: Outcomes
Outcomes are changes, benefits, learnings, effects resulting from your activities. They are the difference in situation between what would have happened without the intervention of your nonprofit, and what was actually achieved with it. Outcomes are usually described as primary (in the lives of the direct service-user, and as a reasonably direct consequence of the service or product) or secondary.
Your outcomes should describe single changes rather than composite ones (which are more difficult to measure), and you should try to steer away from your describing complex changes (e.g. improved well-being).
E.g. Improved literacy skills, increased job search skills, or increased confidence in interviews.
Do: Brainstorm outcomes that make sense for your nonprofit, whether those are changes in attitude or other benefits to the individual or a group.
Step 5: Social Impact
Social impact is the social change, both long-term and short-term, that arises as a result of your activities.
Like the outcomes, social change needs to take into account both positive and negative changes, and both intended and unintended consequences, and both effects upon the intended beneficiaries and on others.
Some practitioners don’t differentiate between outcomes and social impact, however many note a difference between the two:
Outcomes are the short-term and intermediate changes that occur in learners, program participants, visitors, etc. as a direct result of the activity, program, or experience.
Impacts are the broader changes that occur within the community, organization, society, or environment as a result of program outcomes.
Centre for Nonprofit Management states that “While the outcome evaluation tells us what kind of change has occurred, an impact evaluation paints a picture as to how a program might have affected participants’ lives on a broader scale.”
Step 6: Define Indicators
Note: You can do this step in parallel with other steps, or at the very end.
“Data is like garbage. You’d better know what you are going to do with it before you collect it.” — Mark Twain
Indicators, similarly to KPIs, are specific pieces of information. When it comes to impact measurement, they show whether something has taken place and to what extent (e.g. the number of participants that access employment 6 months after the programme).
If you don’t know where to start, look at indicators that similar nonprofits have already developed. However, be mindful of your specific context. Self-confidence indicators developed in one country or one community might not be fully applicable to another.
At the very least, you should define your Output, Outcome, and Impact Indicators.
6.1. Output Indicators
These are specific pieces of information about what outputs you delivered and who you delivered them to, and whether your beneficiaries were satisfied.
Output indicators are usually very tangible (and often numerical).
E.g. Number of sessions delivered, the number of people supported, level of satisfaction of the group.
6.2. Outcome Indicators
They are specific, measurable pieces of information that you can collect to keep track of the difference that your activities are making. Outcome indicators essentially tell you whether you’re achieving your outcomes.
They are usually phrased as ‘level of’, ‘number of’, ‘type of’ or ‘how often’. Outcome indicators should not use a word that indicates change, such as ‘increased’ or ‘improved’ (that’s reserved for outcomes themselves).
It is good practice to set at least two indicators to measure each outcome. Try to combine quantitative with qualitative data, and also subjective and objective indicators (e.g. asking young people whether they feel confident to apply for jobs = subjective, and measuring the number of successful job applications = objective).
E.g. if the outcome is “improved literacy skills”, the indicator could be “number of students passing adult Level 2 literacy at B grade or above”.
Tip: If your outcomes are softer (e.g. improved confidence), try to think about how you can tell that someone’s confidence has improved. That way, you can develop “proxy indicators”. For example, readiness to speak in public can indicate improved confidence, so you can measure that.
6.4. Impact Indicators
These will be the most difficult to set. Social impact indicators will need to account for the change that would have happened without your nonprofit, as well as for the work other organizations are doing. However, if you feel that you can measure the longer-term change in a meaningful way, review your impact statement and think about which indicators you would need to collect information against.
Step 7: Review and Prioritize Your Indicators
In Step 6, you might come up with a long list of indicators and a long list of information to collect. At this point, you need to review the list and select the indicators most crucial to your nonprofit or the ones that are the most revealing.
It is unlikely you have the resources to measure every single change. Try to focus and prioritize and pick indicators that:
- are essential to your work;
- unlock other outcomes;
- are helpful for attracting donors or reporting.
Your indicators will be the foundation for your interviews, focus groups, questionnaires etc. so they are important to get right.
Step 8: Develop Data Collection Tools
Which data collection tools you choose to develop will depend on the size of your nonprofit, the issue(s) you’re tackling, your beneficiaries, and your overall context.
The most common tool used for impact measurement are questionnaires, but there are many others to choose from.
To start off, look at the existing tools, but also tweak them and develop your own.
Whichever you choose, make sure that your data collection tools allow for measurement that is:
- relevant: related to the outcomes it is measuring;
- helpful: in meeting the needs of stakeholders’;
- simple: both in how the measurement is made, and in how it is presented;
- understood and accepted: by all relevant stakeholders;
- transparent and well-explained: so that the method by which the measurement is made, and how that relates to the services and outcomes concerned are clear;
- founded on evidence: so that it can be tested and validated.
Step 9: Track Your Indicators Over Time and Collect Data
Now that you have your indicators outlined and your data collection tools developed, it’s time to deploy them.
Before doing so, train your staff on the basic tenants of responsible data collection and management and how to use the tools. Also, make sure to test and pilot the tools before rolling them out.
Finally, use a good data management software that makes working with data easy, and also keeps it safe.
Step 10: Use Your Data
Working on the Theory of Change and the indicators, and then collecting and analyzing the data can be a lengthy and difficult process. To go through all of it and then not use the data would be a big waste of time, money, and resources.
To stop that from happening:
- Select a data analysis tool.
- Review and analyze your data.
- Draw conclusions.
- Display your data.
There are many different ways you can learn from and display your data. Here are some ideas. Use the data in:
- the beginning or the end of a grant application;
- a social media campaign;
- the annual report;
- a newsletter;
- a learning event;
- a collaborative meeting with another nonprofit;
- strategy meetings;
- staff meetings and retreats.
Common Mistakes When Measuring Social Impact
- Too much bureaucracy: Impact measurement should help, not take up time and resources and weigh down staff.
- The inflexibility of tools and frameworks: Social impact measurement tools and frameworks need to be quick to adapt.
- Confusing accountability with hope: Nonprofits should focus on changes and results that they can be held accountable for.
- Living in a bubble: Organizations need to understand the external context in order to develop more realistic frameworks.
- Assuming it’s all done: Impact measurement frameworks need to constantly be re-evaluated and tested.
A lot has been said about measuring impact in this blog post, and there is a lot more to be said, but it’s important to note that measuring impact is not a project, and should not be something that only one person or department in your nonprofit think about.
Measuring impact should be part of your culture.
Developing simple and practical tools to track progress, and sharing results within and outside your nonprofit, you can better equip yourself to advance your mission, secure funding, and deliver value to all stakeholders.
And although measuring impact can feel like a chore, the long-term benefits vastly outweigh the short-term inconveniences. And it isn’t going anywhere anytime soon!
If anything, remember that, as a nonprofit professional you are on a journey of helping others and addressing some of the most pressing social challenges. Social impact measurement isn’t there to purposely make your life more difficult, it’s there to help turn scarce resources into significant improvements in society.