Evaluating large-scale change is inherently challenging. Traditional methods can feel like a poor fit:
- Reducing change to a few performance indicators that miss the bigger picture
- Imposing rigid, top-down or expensive external evaluations that may not support ongoing learning.
Matter of Focus offers something different: our approach helps organisations evaluate complex change in a way that can be embedded into day-to-day work.
This means using existing data and evidence, and supplementing it with meaningful assessment to understand, track and tell the story of the difference they make at scale.
Through consultancy and our software OutNav, we support teams to collaborate well to:
- Build a shared understanding of what change looks like
- Track and make sense of their data in real time
- Report meaningfully on the difference they’re making.
Our approach builds pragmatic theories of change using simple headings that reflect our contribution analysis approach.
Working through these headings together builds an outcome map which forms a framework for evaluation.
Held within OutNav, teams bring evidence, indicators, data and analysis to their outcome maps to build a robust picture of how change is progressing. This also supports learning, reflection and reporting on impact.
Frameworks can be cloned, shared, and adapted – allowing local teams to work to aligned frameworks while managing their own evidence and analysis. Links are maintained with the overarching framework to allow progress, insights and learning to be drawn up at the global level.
Three different approaches to understanding and evaluating change at scale with OutNav:
- A shared framework used consistently across sites (Scottish Care)
- A national framework adapted locally (Suicide Prevention Scotland)
- A set of nested frameworks aligned from service to strategic level (Midlothian Health and Social Care Partnership)
Scottish Care implements a shared evaluation framework across multiple sites
Scottish Care is the representative body for independent social care services in Scotland. It works with over 400 organisations and nearly 900 services across care homes, nursing, day services, housing support and care at home.
The challenge
The organisation needed a way to capture the impact of its diverse activities – locally and nationally – in a clear and consistent way.
What we did
We supported Scottish Care to implement a co-produced framework that all teams working across local and national levels then used. The aim was to ensure that all teams were aligned in their approach to evaluation, using a shared understanding of outcomes and pathways to impact.
Held within OutNav as the template, the co-produced framework was cloned to all teams working across local and national levels. Each team’s project in OutNav retains a link to the template project, adopting any changes made at the template level.
OutNav’s clone and link feature creates identical copies of the framework ready for teams to work in:
An extract of Scottish Care’s shared framework highlighting the agreed core activities and high-level outcomes (the central stepping stones have been excluded) :
Each team was supported to implement the shared framework using OutNav. Through our collective support workshops, the teams came together regularly share progress, add data and reflect on learning. This approach helped build confidence and consistency in how the framework was used.
Alongside this, Scottish Care developed a clear reporting approach to ensure that everyone knew when reports would be run in OutNav and data and information aggregated and summarised to feed into reporting to the Board.
A small sub team was formed to lead this work, including an OutNav champion who led on the summarising of insights from across teams in a central outcome map and offered informal support between reporting cycles.
A collective analysis support session
The difference this has made
Scottish Care can now report to its board more clearly on the impact of its work – showing how activities at both national and local levels contribute to overarching goals and agreed outcomes. Regular analysis and reporting cycles mean that data feeds directly into planning and learning.
Why this shared framework approach worked well
The teams were already doing similar work and were open to collaboration. Having one shared framework meant it was easier to compare and aggregate results to see the bigger picture, while keeping reporting simple and meaningful.
Suicide Prevention Scotland uses a national framework adapted locally to support evaluation across diverse contexts
Suicide Prevention Scotland drives Scotland’s suicide prevention strategy Creating Hope Together, which brings together multiple organisations working to prevent suicide and address the inequalities that contribute to it.
The challenge
The programme needed a way to evaluate at national and local levels. This meant balancing consistency across the whole programme with the flexibility for teams to reflect their own priorities and ways of working.
What we did
We brought together people from 40 different organisations involved in Suicide Prevention Scotland in a series of facilitated workshops to co-produce an overarching outcome map that reflected the shared purpose and goals of the strategy.
Within OutNav, the overarching map was cloned to each individual project team, who were supported to adapt the national framework to reflect their local context. Through OutNav, they were able to select and refine the pathways most relevant to their work, while still contributing to a shared story of change.
OutNav’s clone and link feature creates identical copies of the framework ready for teams adapt.
An example pathway from the Creating Hope Together outcome map supporting the evaluation of policy making activities:
Ongoing support helped teams embed the framework into their work and use OutNav to track and reflect on progress. From OutNav they reported on progress in ways that made sense for their work while contributing to annual collective reporting, which allowed local insights to feed into a national picture of impact.
The difference this has made
For the first time, Suicide Prevention Scotland can report across the whole programme in an outcome-focused way. The approach has enabled robust monitoring and evaluation and generated learning at both local and national levels, helping to build a clearer understanding of the cumulative impact of the work.
Delivery leads tell us that using OutNav helps them focus more on what difference their work will make, which in turn helps them plan and deliver activities in more efficient and effective ways.
Using OutNav has streamlined communication across the collective, providing a single source for all members to see what is happening across workstreams. This has been particularly helpful for the policy makers who need to provide programme updates within very tight timeframes.
Why this national framework–adapted at local level approach worked well
This approach supports local innovation and responsiveness while ensuring national consistency. Each team can evaluate in a way that makes sense for them, while contributing to shared goals.
It has helped build evaluation capacity within teams, supported innovation, and encouraged a culture of learning and reflection across the programme.
Midlothian Health and Social Care Partnership uses nested, multi-level frameworks to evaluate service-level and strategic-level work
The challenge
The Midlothian Health and Social Care Partnership wanted a system to evaluate how its services contribute to strategic aims and national outcomes.
With work happening across many different teams and services, the challenge was to build a joined-up evaluation approach that supported both day-to-day service improvement and strategic oversight.
What we did
We supported the Partnership to develop a nested framework that integrates evaluation with quality assurance and performance monitoring at service and strategic levels.
At the service level, teams use the same framework to evaluate the impact of their work. This framework links directly to the strategic aims of the Partnership and national outcomes, creating a clear line of sight from service delivery to policy-level impact.
Teams received our ongoing support to embed this approach into their everyday work. Through OutNav they were able to analyse their own data and contribute to a shared view of how services were working towards outcomes.
Alongside this the Partnership developed an outcome map for use at strategic level by the Board. This map brought together data from the use of OutNav by services, alongside key governance information to show how the partnership as a whole is contributing to improved outcomes.
The difference this has made
Teams now use a consistent process to collect and make sense of data at the service level. OutNav enables them to understand what’s working, reflect on what could be improved, and report meaningfully on their progress.
Strategic leads can view aggregated data across services, giving them insight into system-level performance and helping them make informed decisions about where to focus next.
These developments have been widely recognised as pioneering. The work was featured as a case study by Audit Scotland and has provided the basis of the work the Scottish Government is doing to develop the Scottish Learning and Improvement Framework. This tool is currently in development and will be hosted in OutNav.
Why this nested-linked framework approach worked well
This nested approach is particularly powerful in supporting governance, allowing both operational and strategic stakeholders to engage with the data in meaningful ways.
The approach brings together different types of evidence in one place and supports understanding, learning and accountability. It helps manage complexity across multiple services and outcomes while retaining a focus on people and real-world impact.