At the forefront of the COVID-19 response

Community organisations, such as the members of Edinburgh Community Health Forum, have been at the forefront of the response to the COVID-19 pandemic, mitigating the negative impact of COVID-19 on health inequalities. 

Over a matter of days, many organisations transformed their ways of working. They responded to urgent need in their communities and adapted existing services and projects so they could continue to provide support at a distance.

There is so much to learn from this time of rapid innovation. We were pleased to be able to work alongside Edinburgh Community Health Forum member organisations between May and October 2020, to capture learning from their responses to the COVID-19 pandemic.

This work builds on our work together over the past two years. The findings from this work are summarised in this report.

Read the report

The Contribution of Edinburgh Community Health Forum Member Organisations to the COVID-19 Response

Download (pdf)

Our evaluation approach 

The evaluation was carried out using the Matter of Focus approach, a tried and tested, theory based approach to understanding the contribution organisations make to improved outcomes.  

The approach is specifically designed to evaluate complex, people-based work, such as that carried out by community organisations, where it is not possible to simply measure the difference made.  


Text reads: An overview of the Matter of Focus approach

To learn more about the Matter of Focus approach start with our overview Insight post. Go to post.


The House of Care Programme is a partnership between:

The House of Care Programme aims to transform primary care to focus on what matters to people. It is part of a family of approaches seeking to shift health and social care to more preventative, personal outcome focused, person centred and asset-based approaches. We are delighted to share our final evaluation report from this work.

picture of a report front cover

Read the report

From fixer to facilitator: evaluation of the House of Care Programme in Scotland

Download (pdf)

Our evaluation approach

The evaluation was carried out using the Matter of Focus approach, a tried and tested, theory based approach to understanding the contribution a programme has made to improved outcomes.

The approach is specifically designed to evaluate complex, people-based interventions such as the House of Care where it is not possible to simply measure the difference made.

Sharing the findings

In October 2020 we hosted an interactive webinar to hear from different stakeholders involved in the evaluation. Watch the recording here:


You may also be interested to watch the recording of a webinar we hosted with Lothian House of Care in March 2020, in which we talk about the findings from their local evaluation, also carried out using the Matter of Focus approach. Watch the recording here.


An appropriate focus on outcomes helps organisations and individuals manage complex projects and programmes and respond appropriately to the challenges that inevitably arise.  Using outcomes well creates a framework for action.

In this post, I outline four ways in which using outcomes as a framework for action will empower your work towards social change and when to be cautious with outcomes.

Four ways outcomes help with change:

1. Build a clear and shared vision of where you want to get to. 

a dotted line and marker

Effective change programmes are driven by the outcomes that matter to the people and communities at their heart. The first step on the change journey is to agree what matters to people and why it matters.

2. Plan how you can work together to make the change you hope for.

A focus on outcomes gives you a starting point to agree a logical (though not necessarily linear or simplistic) understanding of how your planned activities will contribute to the difference you hope for. Breaking the change process down into meaningful steps enables you to agree the contributions of everyone involved and how you expect to get there.  

3. Track your progress towards the outcomes you care about.

coloured blocks representing tract progress

When delivering people-based services and programmes to address complex social issues, change always takes time and often involves dealing with and responding to the unexpected, both good and bad.

A meaningful outcomes framework provides something tangible to anchor your work, track progress against and helps you flex your plans as required.  It can help you see when and where progress is slow, and give you tools for regrouping around what matters.

4. Reflect on the process and share and embed learning .

Capturing learning and telling the story of complex change processes can be hard, especially when you want to capture what went well and where things were challenging.

Having a clear framework for how you hoped your work would contribute to outcomes helps make sense of your experience and evidence. It also helps you tell a realistic story that others can learn from and gives space to highlight the unique contribution that individuals and organisations made towards the changes that were realised.  

A note of caution…

Whilst we think good outcome focused tools and approaches can make great frameworks for action, there are a few things they are not good for. 

They are not good for – measuring or attributing change, assessing performance or for managing resources and rewards.  This is because when we are doing complex, people-based change our work is only ever one factor that influences whether the outcomes we hope for are realised. See our post on why data on children’s wellbeing can be misleading for more on this.

When working with outcomes we must always remember that we do deliver services, we don’t deliver outcomes – we work towards them.

Putting meaning at the centre of your work

In Scotland that are lots of organisations that have been working in an outcome-focused way for many years. Evidence tells us that people enjoy working in this way – that it’s satisfying, and realistic.

Meaningful, useful and effective outcome approaches empower the people that are making change happen by:

when-sarah-met-ailsa

For both of us, this was our first foray into the world of business, motivated by the desire to have a more tangible positive impact on the world (see Sarah’s post about the journey from academia to mission-led start-up ).

We had both spent our careers working with and in public service organisations supporting them to make better use of data and information to learn and improve as well as researching the factors that got in the way.

Having developed a solution that we passionately believed could overcome many of the barriers we had learned about, Sarah and I were keen to find a way to get it out to as many organisations that could benefit as possible.  And so, it was with great excitement that first Matter of Focus and then our software, OutNav, launched onto the burgeoning Edinburgh tech start-up scene.   

One of the key mantras of our work with public services, is ‘It’s not what you do, but the way that you do it’. 

It is one thing for organisations to say that they are collaborating, empowering, involving or engaging – the real test is how people act day in and day out to bring this to life.

From the start we brought the same thinking to our business. It is not enough for us to talk a good game, the true test of whether we are sticking to our claims and credentials as a mission-led business is to be found in those many small practices and decisions we take every day.   

Embracing opportunities

Over the past two years we have enthusiastically embraced all the opportunities we can to learn about business and have systematically sifted through the books, the advice and models and extracted the ones that are useful for us including support from Business Gateway and Scottish Enterprise.  

We have always valued to the opportunity to have our business model and strategy interrogated by people with many decades of experience of working with large and profitable businesses, but we have found that there have been relatively few bits of advice that we have felt comfortable adopting in the context of our mission-led business. 

a dotted line and marker

Instead, we have found our own way of addressing the various challenges that any ambitious and growing business faces, linking with like-minded businesses from the Zebra and B-Corporation movements. In so doing we have often come back to our long experiences of supporting and spreading innovation in public services.  

This has led us to hone down on a few key principles that guide the work of Matter of Focus.

Key principles that guide our work

1. Only work where we can deliver value.   

Organisations with a vision for social change internationally are under great financial pressures. We recognise that when our clients spend money with us it is money not spent directly helping the people and communities they care about. We have found that by adopting our approach client organisations have become more efficient, effective and better able to leverage funding. 

Through continuously learning we understand which kind of organisations are best placed to benefit from our support and use this as the basis of discussion with any prospective clients. This ensures that all of us are clear about what our work together will involve and are as confident as we can be that it will bring value to the organisation.   

2. No one size fits all.  

Standard business advice encourages start-ups to create a single (ideally simple) product, identify who needs it and then make sure as many of those people are reached as possible. This works well if your aim is to make money, but not if your aim is to contribute to meaningful change. 

We are excited and proud that many organisations are successfully using our software product OutNav to learn, improve and tell their story. But no single product is going to deliver for the many organisations who could benefit from our support on meaningful outcome evaluation, because there is no simple fix for such a complex problem. Over the next year we will be building a diverse range of products and services to meet a wider variety of needs. This will include training on working with outcomes, data, evidence and evaluation. 

3. Create the conditions for change 

Our mission will be complete when people don’t need us anymore. We dream of a world where organisations with a vision for social change have the tools and capacity to continually learn and improve. In this ideal world, commissioning and performance management structures would value learning and improvement over crass ideas of performance. Mainstream improvement organisations would be able support meaningful outcome working and evaluation both practically and technically. We feel our biggest contribution to creating this change is by sharing the message that meaningful outcome working is possible and brings many benefits. We do this through our blogs, seminars, webinars and reports. We actively work with our clients to influence decision makers in their local systems to raise awareness about the benefits of putting learning front and centre.  

what are outcomes - ailsa cook
Delivering our free seminar ‘Meaningful outcome evaluation, a practical approach for public services’ to a full house.

4. Be flexible 

A central mantra of profit-led business is standardisation. Standard packages, features, products, timeframes. However, the organisations we work with are incredibly diverse. Going down the route of standardising everything would mean that we miss some key opportunities. We have worked with organisations from very small, to government departments, looking at a broad range of people-based outcome challenges. These have included work focused on professional improvement, children, families, carers and older people. We have learned a lot about the fundamentals of planning for, learning about and evaluating outcomes that a more standard approach would not have delivered.  Our clients include organisations delivering services, improvement organisations looking at whole sectors or issues, and researchers and evaluators. Because as a business we have a mission, we need to reach everyone who faces the outcome evaluation challenges we want to address. This means tailoring messages to different people and sectors and developing multiple solutions that meet their specific needs.   

purple dotted line

Looking forward

As we embark on year 3, we’ve reached a point in our journey where our pioneering early adopters are getting great value from our approach and we’ll be telling their stories here over the next few months. We are also really excited about the new clients coming on board over the summer.  

If you want to stay up-to-date with our news and you’re not already part of our growing community, sign up to our mailing list.

A small country making pioneering policy decisions

Uruguay is a small progressive country with a population of 3 million and a history of taking radical policy decisions to reduce inequalities and improve public health.

Uruguay was the first Latin American country to ban smoking in public places. And it successfully beat prosecution by Phillip Morris to uphold its strict laws on cigarette packaging. This action opened the door to the rest of the world to follow suit.  

Another pioneering policy was the decision taken more than 10 years ago to enable every child to access the internet, through the provision of public Wi-Fi in every community in Uruguay, as well as providing free devices so all children had the opportunity to access this Wi-Fi.  

More than 10 years on, every child in Uruguay has the opportunity to access educational and social resources online, as well as to create their own content.

However, whilst there was lots of discussion in the country about the risks and benefits of this for children, until last year there had been little in the way of comprehensive evidence to understand how children behaved online and what they thought about the opportunities and risks.  

The Global Kids Online project

In 2017 Uruguay joined the Global Kids Online project and, drawing on its framework and resources, carried out the first research into kids’ experiences online.

The purpose of my visit was to understand the impact of this research on the development of policy and practice in this area, and ultimately the lives of children and families.  

We are in the process of developing a case study which will share the learning in full. But I wanted to draw out some key messages that I took from my experience in Uruguay:

Understanding evidence to action and the process of evaluating impact  

It’s all about partnership working

At the heart of Uruguay’s approach to the Global Kids Online research was a strong partnership between UNICEF and UNESCO; the two main government agencies, Plan Ceibal and Agesic; and local academics. An overarching finding from our research was how important that partnership was to ensuring evidence got into action.

Cecelia Hughes and Martin Rebour from Plan Ceibal
Introducing Cecelia Hughes and Martin Rebour from Plan Ceibal to the Matter of Focus headings

In talking to the partners about their work together I was frequently reminded of experiences in Scotland.

Partners said their journey together was long and time consuming. They came to the collaboration with different priorities and drivers for being involved. The team included people with a breadth of relevant expertise and there were robust discussions about methodological issues, the focus of the research and the best way to approach analysis, reporting and dissemination.

What came through loud and clear in the research was how passionate everyone was about the project and committed to developing the best outputs and outcomes possible.  

Discussing this partnership with other stakeholders less closely involved, this commitment and willingness to disagree and discuss clearly paid off. Everyone valued the quality of the research and the clarity of the key messages.  

The team took an innovative approach to reporting in partnership. They spent several meetings agreeing the key messages from the research, that were driven most closely from the empirical data. Then each partner had space in the report to reflect on what these findings meant for them and to offer their own interpretations. In this way, the team was able to turn their differences into a key strength. They also followed this through into sharing the findings, developing a media plan that involved each member of the team speaking to their area of strength.  

Spending time with the Unicef Uruguay communications team

What made this partnership possible?  

A key feature of the Uruguayan context that people referred to time and time again was that it is a small country – many of the partners knew each other already. They also knew that they would be likely to work together in some capacity or other in the future. Not only did they have a head start in relationship building, they also knew that they were creating lasting relationships and it was worth getting them right.

Secondly, the partners were united by a shared vision for the research. Everyone agreed that getting a robust understanding of children’s experiences online was vital to the successful implementation of this policy area.  

Thirdly, in developing the Kids Online Uruguay work they were building on a well-established international framework and partnership that all the partners had faith in. This allowed them to short cut some learning and focus on the issues that were most important and particular to them 

The benefits of successful partnership working  

Working in partnership in this way enabled the team to deliver a quality piece of research that all the stakeholders we spoke to felt was robust and valuable. It also meant that the research findings were immediately put into action by the agencies with the most power to act.

All the work that went into building the partnership to carry out the research has strengthened the evidence to action eco-system around these issues. Future research will be carried out in partnership.  

What can we take from this example?  

For me, working in the Scottish public service context, going to Uruguay was a bit like finding a long-lost sister – another progressive nation, with landmark policies to address inequalities.  

The trip reinforced lessons about the importance of getting partnerships right from the start and of embracing the diversity of perspectives that different partners bring to the process. Partnership working is hard, takes time, but done well can deliver great results.

This post captures the learning from a 2019 OutNav Community meeting that focused on embedding the practice of meaningful outcome evaluation across organisations.

Our clients have overcome many practical and conceptual challenges, like reconciling the outcomes of their work with those specified by funders and working out mechanisms for staff to systematically reflect on practice. In so doing, these organisations are cementing their place at the forefront of best practice internationally in meaningful outcome evaluation.

We are grateful to two of our clients, Julie Gardner from the Thistle Foundation (discussing her work with Lothian House of Care) and Franca MacLeod from the Scottish Governments’ Children and Young People’s Improvement Collaborative, for sharing their work with the group.

Embedding reflective practice

Julie Gardner opened the morning by sharing the work of Lothian House of Care team to embed reflective practice in their work.

Julie Gardner House of Care presents to the OutNav Community

Time and space

As a team, Lothian House of Care believe that giving practitioners the time and space to reflect on their practice is critical to enabling them to work in new and more person-centred ways. In addition, the team recognise the value of this as evidence to inform their evaluation of the programme.

Lothian House of Care has taken a flexible approach to making space for reflective practice. For example, creating protected time for reflection in training sessions, incorporating reflective practice in team meetings and even holding lunch clubs to enable staff to reflect.

Focus

In order to keep reflections consistent, the team use a set of questions to focus their discussions:

  1. What are you noticing?
  2. What has been better?
  3. What is working?
  4. How did you do that?
  5. What are you learning?
  6. What will you be pleased to notice in the coming weeks?

Capturing reflections

The staff reflections are captured in a variety of ways so that they can be used by the Lothian House of Care Team to contribute as evidence to their ongoing evaluation.

In some cases reflective notes are captured on a flip chart and then photographed, at other times staff are asked to write their reflections in a reflective impact log. Reflections are also captured by other team members as they write up meetings and sessions. These sources of evidence can then be uploaded to OutNav and used to inform the evaluation.

The difference reflective practice makes to the team

Julie made a strong case for the value of reflective practice as a tool in itself as well as to contribute to evaluation. She summarised the difference that reflective practice makes to the team:

The time invested in reflective practice at Lothian House of Care has resulted in lots of smiles.

Julie Gardner, Lothian House of Care

The benefits of using OutNav to embed evaluation in improvement

Franca MacLeod shared some of the work they have been doing using OutNav to evaluate a Scottish Government improvement programme, the Children and Young People’s Improvement Collaborative, CYPIC.

Franca Macleaod Children and Young People's Improvement Collaborative presents to the OutNav Community

Franca highlighted three aspects of working with OutNav that she found particularly helpful:

A clear focus on activities and engagement

As with many improvement programmes, the work of the improvement advisors is complex and diverse. Going through the process of outcome mapping has enabled them to develop a very clear and shared understanding of what they do and who with, to underpin the evaluation.

this approach allows you to be closer to the activities and engagement.

Incorporating different kinds of data and feedback

When evaluating improvement it can be hard to gather as much data as you would ideally like in a robust and systematic way. Franca highlighted the benefits to them of using OutNav to incorporate the observations and reflections of improvement advisors. Most importantly from a research perspective, being able to assess confidence in the evidence allows CYPIC to include less robustly collected data and highlight that there are limitations.

Capturing data and feedback directly from improvement advisors

CYPIC are making the most of the flexibility afforded by OutNav and have set every improvement advisor up on OutNav with their own project. This has filled an important gap for them, enabling each improvement advisor to gather and analyse the management information that they need to ensure their projects are progressing well. Holding this in OutNav gives Franca a single space where she can easily access all the data and use it to inform the evaluation.


Text reads: Track your progress and impact in OutNav

If you would like to know more about how evidence is rated within OutNav, read our post Track your progress and impact in OutNav.


We were delighted to be joined by a full house of 70 people from a wide range of organisations from the Scottish Government, to smaller community organisations. It was great to have participation from elected members as well as clinicians, evidence and evaluation specialists, planners, commissioners, policy makers and people managing and delivering services.

It was clear from the turnout and the volume of conversation in the room that outcome evaluation is something that touches and engages people right across public services.  

Ailsa and Sarah shared the conceptual and practical approaches we take to supporting organisations in their meaningful outcome evaluation.

Key messages: 

We need new tools and frameworks to help us deliver and improve the collaborative, personalised and co-produced public services of the future.

Many of our current outcome evaluation approaches are based on work from the 1960s and are not fit for purpose. We know that using them inappropriately can undermine good outcome focused working.  

We see meaningful outcome evaluation as a social practice, that needs to be underpinned by appropriate meanings, knowledge, competencies and skills as well as the right materials and tools.

Many of the organisations we work with are caught between paradigms, wanting to work in relational ways to learn and improve, whilst still being required to account for their performance.  

When working in complex systems, it doesn’t make sense to try to attribute changes in outcomes to any given intervention.

Public services can only ever make a contribution towards improving outcomes alongside other factors. Theory based approaches to evaluation, such as ours, enable organisations to plan for and track how their programmes make this contribution.   

There are a number of strategies that you can use to ensure your outcome evaluation approach is inclusive and generates insights that help your organisation to learn and improve.

Strategies we use include: starting with a clear understanding of the context in which you are working, using plan language, taking a collaborative and inclusive approach to mapping contribution to outcomes, being clear about the mechanisms for change and involving everyone in data gathering and analysis.

We believe that meaningful outcome evaluation is a central to improving and reforming public services.

Making the shift to more meaningful work with outcomes involves letting go of some bad habits, as well as embedding new practices. Funders, commissioners, government and scrutiny bodies are key to enabling everyone to make the shift to more meaningful outcome evaluation. 

Questions from the audience

The seminar finished with some brilliant questions and discussions from the audience covering issues such as outcome-based commissioning, gaming and performance management, embedded vs external evaluation and outcome measurement. Participants then joined us for a glass of fizz to celebrate the launch of our new brand 

If you attended this seminar and would like to discuss any of the issues we explored further, do get in touch.

We know that for an organisation wrestling with challenges such as how to streamline data collection or make sense of their social media analytics, nothing beats the chance to learn from others who have been there before.

It has always been a priority for us to establish a strong community of users around our approach. Therefore, we were delighted to welcome a pioneering group of OutNav users to our second OutNav Community get together in Codebase on the 30th August 2018.

It is just over six months since we gave our first group of beta testers from StarcatchersHealthcare Improvement Scotland and the Thistle Foundation access to OutNav. Since then, we have been joined by more beta testers and together we have made significant progress on our journey to create the first software product that enables non-expert researchers to plan and implement robust and meaningful outcome evaluation using a theory based approach.

It has been inspiring to witness the innovative work our clients have taken forward to embed meaningful outcome evaluation practices within their day to day work.

Our clients have overcome numerous practical, political and conceptual challenges, like reconciling the outcomes of their work with those specified by funders and working out mechanisms for staff to systematically reflect on practice. In so doing these organisations are cementing their place at the forefront of best practice internationally in meaningful outcome evaluation.

The focus of this meeting of the OutNav Community was on sharing practice.

Pioneering OutNav users from three organisations, presented their work to date. Fraser Stone from Starcatchers presented his report from the Creative Kin project evaluation he ran from start to finish in OutNav. Tabitha Casey from the University of Edinburgh demonstrated her use of OutNav to complete the final analysis and report on the Partnership Research to Tackle Violence Against Children in Zimbabwe project. Ashley Spalding from Stirling Council talked about her plans to use OutNav for the evaluation of the Neighbourhood Care project. The community enjoyed engaging with, and reflecting on the issues and learning from these projects.

Matter-of-Focus Event

One of the most overwhelming issues emerging from the discussions was the time that it took to become confident in outcome evaluation.

Everyone in the room had stories to tell of early mistakes, be that in planning, analysing or engaging colleagues.

People also had masses of learning from the early months of their work in this way. Some of the key messages and issues are explored here:

The value of meaningful outcome stories

Whilst talk of outcomes is everywhere across UK public services and beyond, the reality is that very few projects or organisations meaningfully report on outcomes. Common approaches to outcome reporting focus solely on the outcomes ‘achieved’ and in doing so mask the complexity and value of the work undertaken. Whilst most people at the meeting had been working with the approach and software for some time, this approach to reporting is new and they all found seeing other peoples’ outcome reports created through OutNav inspiring. As one person said, “it is really helpful to see the story”. It was particularly valuable to see different approaches others had taken to incorporate photos and graphics into their reports, really bringing the story alive. Other participants shared positive feedback they had already received on the coherence of their outcome-based reporting.

Getting people on board

OutNav is a tool that enables organisations to embed evaluation into everyday work. Whilst being able to draw on external or specialist research is often very helpful, we believe that making evaluation everyone’s job is critical to ensuring that the learning from any process of evaluation is continually translated into improvement. A key issue discussed at this OutNav Community meeting was how to get people on board with evaluation early in the process. Effective strategies discussed included:

Understanding context

Participants highlighted the complexity of their projects and the multiple factors that influence whether or not their projects and programmes will make an effective contribution to improving outcomes. For many of these projects, small improvements in knowledge, confidence, skills or behaviours for some of the people they work with, constitutes a big success. Other projects are very vulnerable to factors out with their control undoing all of their good work, for example cuts to mainstream services and or benefits. Participants highlighted the importance of making these challenges to projects explicit early on. This is important to ensure that:

To help organisations achieve this we often use the ‘ISM Behaviour Change Framework’ a toolkit developed by the Scottish Government to promote environmental behaviour change, but that can be used for any change project to understand how factors operating in the individual, social and material contexts are likely to influence success. Participants were particularly interested in this tool and in hearing how others had used it alongside the outcome maps to analyse context.

Find a champion (or two)

A key reflection from the morning was the importance of having one or more people champion outcome evaluation within each organisation. Participants highlighted the many different time pressures everyone faces and how easy it is for evaluation of any kind to slip down the agenda. They also reflected the need to engage with the process regularly to keep up confidence and skills in driving forward the evaluative work.

Investing time in OutNav is time well spent

Using OutNav creates new opportunities for outcome evaluation, for example the ability to share good evaluative practice between projects. Participants reflected that the more time they spent in OutNav the more they benefitted from its accessible and structured approach and its collaborative and practical functionality.

For us here at Matter of Focus this second OutNav Community meeting felt like a really important milestone. We have spent the past year and a half planning for and building a tool that reflects the complexity of public service evaluation and supports non specialist researchers to carry out great outcome evaluation in a way that embraces, rather than reduces, that complexity. Along the way we have had moments of uncertainty and questions about whether we have bitten off more than we can chew. Through all of this the commitment and inspiration of our amazing pioneering clients has kept us focused and on track. We are very excited about what we can achieve together over the coming months and years!

Earlier this year I was at an event organised by the Alliance for Useful Evidence which highlighted how central outcomes-based approaches were to all four administrations of the UK1. Scotland, Wales and Northern Ireland have all engaged in significant exercises to map out the outcomes that are important to their populations and to develop measures and indicators to see whether they are on track.

In Scotland, the concept of outcomes is interwoven with the ongoing drive to reform public services through a focus on efficiency, accountability, partnership, personalisation and co-production.  But why? How can a focus on outcomes help with these things?

We believe that focusing on outcomes helps organisations, projects, funded programmes, services and even governments achieve three things:

  1. Communicate what is important
  2. Learn and improve
  3. Be accountable

 

Communicate what is important

A clear and shared purpose is critical to delivering on complex and collaborative projects, from small initiatives to whole programmes of government. Communicating the outcomes you want to achieve at the start helps everyone stay focused on the task at hand and shows how the work of different partners contribute to the whole. Indeed, having clear and shared aims and vision has been identified as vital to effective partnership working (see the What Works Scotland Review of Partnership working).

Learn and improve

Projects rarely unfold quite as intended. For projects to be successful, teams need to regularly monitor progress, identify blockages or opportunities and flex their approach in response. Having a clear ‘theory of change’ that articulates how you think your activities will achieve your desired outcomes and what needs to be in place to make this happen, provides a foil for this reflection and learning. For example, in our work with a health and social care project, getting their whole team together to review the evidence as to whether they were on progress to realise their outcomes identified numerous opportunities to improve the service and the way they went about evidencing their work.

Be accountable

In a context of austerity, projects are increasingly being asked not just to demonstrate what they do, but also the difference they make. Outcome measurement for accountability was first introduced in the 1950s and ‘60s in the USA where major funders of social projects, like the Kelloggs Foundation, wanted to find out more about what happened to the people who used the projects they funded. In recent years it has spread to local and central government in the UK.

All four UK administrations now have a suite of national indicators and measures so they can report performance and be accountable to the public. Services and funded projects are also commonly asked to report on the outcomes of their work and in some cases payment is linked to achievement of these outcomes, for example through Payment by Result schemes.

However, a warning. Of the three reasons for focusing on outcomes, accountability is the trickiest.

Many public service projects are delivered in complex systems where many factors influence final outcomes. In this context, holding projects accountable for delivering outcomes over which they have limited control can lead to a raft of unintended consequences.2

Using a theory based approach, such as our Matter of Focus approach, that shows how projects contribute to improving outcomes can help organisations overcome these issues. We will explore issues around outcome measurement and understanding cause and effect in a later post.

This is not surprising given the lack of consistency in the way the term is used across policy, practice and research. As a result it has become a concept that everyone uses, but few truly understand, which can make it hard to put outcomes into practice.

The three types of outcomes

Programme level outcomes

Outcomes are most commonly defined as the end result or impact of a service, programme or intervention. Outcomes can be:

We refer to these kinds of outcomes as programme level outcomes.

Population outcomes

Many national and local organisations also refer to outcomes when they are describing or measuring how things are for their population.

Population outcomes are not outcomes in the truest sense of the word as they are not directly linked to any specific  activity or intervention. Instead they are a description of how things are for people living in this place / country, or how the government would like them to be.

Personal Outcomes

A final way in which the term outcome is used is to talk about Personal Outcomes, which describe the things that matter to a person. Taking a ‘personal outcomes approach’ (such as the Talking Points approach1) involves having a good conversation with the person that explores their hopes for the future and creates a plan for how this can be achieved, drawing on the person’s own assets as well as formal supports where needed.

Personal outcomes are distinct from the outcomes for individuals of a programme or intervention, which may or not be things that matter to a person. For example, going to a day centre may increase the person’s social contact, but not with people they value spending time with.

The a-ha! moment

Here is a slide which often gets an ‘a-ha!’ reaction when we present it:

An outcome is the end result of a chain of events that starts with inputs and processes, leads to an output (in this case, the birthday cake) which then leads to an outcome (happy children). What we’re talking about when we’re focusing on the outcome is the happy child, not the cake.

Getting your head around outcomes can take time2. We have learnt, however, that once people have the ‘light bulb moment’ and can think confidently about outcomes, it helps solve seemingly intractable problems at work and in the rest of life. The simple exercise of asking questions like ‘Why are we doing this? What do we hope to achieve? What difference will this make?’ opens the door to new possibilities and the understanding and insight with which to make better decisions.

The value of starting with the end in mind

Here at Matter of Focus we are clearly outcome enthusiasts, but we’re not alone in thinking this way. The value of ‘starting with the end in mind’3 is recognised internationally and promoted by governments across the UK, North America, Australia and New Zealand; as well as key international actors like the World Bank, the EU and the OECD. So wherever you are in the world, if you are working in public services, the chances are if you are not already asked to report on, plan for and evaluate outcomes, you might be sometime soon.

If you are keen to read more, here is a pdf copy of a position paper I wrote for What Works ScotlandOutcomes Based Approaches in Public Service Reform


Why focus on outcomes? We explore what a focus on outcomes helps organisations, projects, funded programmes, services and even governments achieve. Why focus on outcomes?


Notes

1 The Talking Points: Personal Outcomes Approach was developed in Scotland and is used by organisations internationally. For more information and resources visit the personal outcome network website.
2 A finding echoed in this interesting report from the World Bank
3 Friedman, M (2005) Trying Hard isn’t good enough’. Fiscal Policy Studies Institute.