One of the golden rules for effective evaluation is to work out how you will analyse data, before you start to gather it. In our years of supporting practitioners to carry out research and evaluation, we’ve noticed that people struggle with planning for and doing analysis.
Many of our clients feel that they have loads of data they are not using effectively. Sometimes they have bags of feedback forms, flip charts and sticky notes, with little plan of what to do with them, or a virtual version of the same problem with survey responses, feedback and evaluation in files on different computers. It is a common issue: individuals and organisations spend lots of time collecting data, but then don’t know what to do with it.
One of the things we love most about outcome-based evaluation is that, done well, it completely simplifies the analysis process.
Outcome based evaluation keeps you focused
Our approach starts with setting out what needs to happen for your project or programme to work. We do this through a theory-of-change approach we call outcome mapping, using our headings. We help organisations create one or more outcome maps as the basis for understanding, tracking and evaluating their progress.
The purpose of an outcome focused evaluation is to determine if your project is unfolding as you anticipated and if so what is helping; if not, what is getting in the way. This brings a real focus to your data collection and analysis. For example, your outcome map will set out exactly who you need to engage to ensure your project or programme is effective. This then provides a focus for what information you need about engagement to understand if you have reached the right people, and a way of talking about what happened when you analyse that data.
Outcome based evaluation helps you tell meaningful stories about your work
Analysing your data, information and feedback is fundamentally a process of making sense of your data and then telling the story of it in a robust and transparent way. Defining specific pathways through your outcome map provides a clear structure for those stories- following the headings, making it easy to see how different aspects of data and evidence fit together to help you understand how the project or intervention has engaged people, what they have learned and gained, and how this makes a difference.
Outcome based evaluation breaks analysis down into achievable chunks
Outcome based approaches, like ours, plot a project’s contribution to outcomes into discrete items that we call ‘stepping stones’, under our headings. Each stepping stone needs to be in place for you to claim that your project has made the contribution to outcomes you anticipate. Outcomes based analysis simply involves reviewing the data and information relating to each of these steps individually and assessing whether there is good evidence of progress for each step. The qualitative and quantitative data that relates to each step can then be summarised in narrative and/or graphic form.
Breaking analysis down into these manageable chunks is hugely beneficial if you’re doing evaluation alongside your day job, as unlike more traditional approaches to analysis, progress can be made in short sharp bursts.
Our top tips for great analysis:
- Show your workings: be clear about where the information has come from and how you have used it to reach your findings.
- Remember you are telling a story: this is to someone who might not know your research or project. Include all relevant details and use it as an opportunity to share your insights and learning about what you learnt from the process.
- Be reflective: are you bringing a particular perspective or bias to your analysis, would other people reviewing this data reach the same conclusions?
- Use success criteria: defining what success looks like helps to structure the process and keep you focused.
- Keep it sharp: no-one wants to read more than they have to, so keep it to the point.
- Bring your findings to life: use examples, though do bear in mind how representative they are.