Evaluating impact
Impact is not a feeling.
How to distinguish strong evidence from compelling storytelling — and what to do when the data simply isn't there.
Outputs. Outcomes. Impact. They're not the same.
The mistake that catches trustees most often is confusing a large output number for an outcome. A charity reports that 4,200 people attended their financial literacy workshops last year. That's an output — it tells you the sessions ran. Whether any of those 4,200 people now manage their money differently, carry less debt, or make better financial decisions isn't in that number. If the trustee meeting moved on because 4,200 sounded impressive, the outcome question was never asked.
Outputs
The direct, measurable products of an organisation's activities.
The limitation
Outputs confirm delivery. They tell you something happened, not that it mattered.
Outcomes
The changes that occur in people's lives, attitudes, behaviours, or circumstances.
The limitation
Outcomes require measurement over time, and attribution to the programme rather than other factors is rarely straightforward.
Impact
The portion of outcomes that are attributable to the organisation's work, net of what would have happened anyway.
The limitation
True impact is extremely difficult to measure with rigour outside of randomised controlled trials — which are rarely appropriate for most charitable work.
What counts as evidence — and how to grade it.
Testimonials are vivid, immediate, and easy to read in a trustee pack. A case study of someone whose life changed is more persuasive than a pre/post measurement table, even though the table is more reliable. The problem is that the case studies in a charity's annual report are selected by the charity's communications team — they're not a sample, they're a highlight reel. The hierarchy below isn't about dismissing the vivid stuff. It's about knowing what it can and can't tell you.
Randomised controlled trials (RCTs)
Gold standard for establishing causal attribution. Compares outcomes in a treatment group versus a randomly assigned control group. Rare in the charitable sector; appropriate mainly for replicable direct service interventions.
Quasi-experimental studies
Comparison group selected on matching criteria rather than random assignment. Common in health and education evaluation. Reduces — but doesn't eliminate — selection bias.
Pre/post measurement
Compares a beneficiary group's status before and after an intervention. Does not control for other factors that may have caused change, but is practical and widely used.
Validated survey instruments
Scales and questionnaires developed and validated by researchers — such as Warwick-Edinburgh Mental Wellbeing Scale — carry more weight than bespoke surveys.
Qualitative research
Interviews, focus groups, case studies. Weak for measuring scale; strong for understanding how and why change occurs. Essential complement to quantitative evidence.
Administrative data
Records generated by the organisation's operations: referral data, session attendance, case progression. Useful for outputs and proxies; rarely sufficient alone for outcomes.
Beneficiary testimonials
Valuable for illustration and communication. Not sufficient as primary impact evidence — testimonials are self-selecting and provide no systematic basis for comparison.
SROI and its limits.
What SROI is
A framework for measuring and communicating the value created by an organisation relative to the resources invested. It produces a ratio: for every £1 invested, £X of value is created.
Its appeal
Reduces impact to a single number that funders can compare and communicate. Seems to provide an objective basis for portfolio prioritisation.
Its limitations
The ratio is highly sensitive to valuation assumptions that are difficult to make robustly — particularly for social and wellbeing outcomes. Two analysts applying SROI to the same programme with reasonable but different assumptions can produce ratios that differ by a factor of three or more.
How to use it
SROI is more useful for understanding which inputs most drive value creation than for comparing organisations. Treat the ratio as indicative, not definitive. Always interrogate the assumptions behind it.
"A single number can obscure more than it reveals."
SROI is most useful when you're comparing different interventions for the same programme, or tracking improvement over time — not when comparing fundamentally different types of charitable work.
Numbers alone don't tell the whole story.
The most useful piece of evidence on one assessment wasn't a number. A programme manager explained that their employment outcomes only held up for participants who stayed engaged for more than eight sessions — anything below that, and people left without the connections that drove the change. That wasn't in the impact report. It was the explanation for why the report's numbers looked the way they did, and it told you exactly what to protect in the next funding conversation.
A programme that reduces reoffending by 18% is interesting. Knowing that it works through mentoring relationships that continue beyond formal programme completion is actionable — it tells you what to protect when funders push for cost reduction.
Look for qualitative evidence that is systematic rather than illustrative. A structured set of interviews with a representative sample tells you something. A curated selection of success stories tells you about curation choices.
What good qualitative evidence looks like
- Systematic sampling — not just willing participants
- Interview or focus group protocols documented
- Negative or mixed findings included, not filtered out
- Findings coded and analysed, not just quoted
How cleargiving.io structures impact evidence.
Here's what goes wrong without structured evidence. A trustee reads a charity's annual report and sees the line: "312 young people achieved improved confidence." That sounds like an outcome — and it might be. But it might also be 312 people who answered yes to "do you feel more confident?" on a survey the charity designed themselves, administered at the end of the session, before they'd gone home and tried to apply anything. The two things look identical on paper. cleargiving.io surfaces where the number came from, who designed the instrument, and whether anyone validated it — so that figure becomes a question to examine rather than a conclusion to accept.
Separates outcome claims from evidence quality
A charity can claim significant outcomes and have weak evidence. cleargiving.io surfaces both the claim and the quality of evidence independently, so trustees understand confidence as well as content.
Flags missing evidence explicitly
If a charity has no outcome measurement data, that absence is recorded as a finding — not filled with a plausible-sounding assumption.
Grades evidence by source
Peer-reviewed evaluation carries more weight than a case study in an annual report. The evidence quality rating reflects this distinction.
Prompts trustee discussion
Where evidence is limited or mixed, the assessment surfaces the question — rather than resolving it. Trustees see open questions that warrant further enquiry before deciding.
Evidence-informed giving starts here.
cleargiving.io surfaces impact evidence, grades it, and flags what's missing — so trustees can make decisions with their eyes open.