Scottish Women’s Aid (SWA) is the lead organisation in Scotland working towards the prevention of domestic abuse. In 2020/21 Scottish Women’s Aid piloted a webchat service providing direct support to children and young people experiencing domestic abuse, involving children and young people in service design.
Matter of Focus acted as the evaluation partner for this work, helping SWA to first map their outcomes, and acting as an external guide to gathering data, reflections and learning against their outcome map.
This work highlights key points of learning and impact and models a ‘highlight findings’ report, based on using the Matter of Focus approach in a practical way to meet the needs of a focused and short-term development project. It was felt to be especially important for this project, as a learning piece, to invest in evaluation to sit alongside the work and to capture the learning as fully as possible.
Jump straight to the report now
Many years’ experience of participation work with children and young people has told Scottish Women’s Aid that children and young people want a service that is just for them, not an extension of adult services.
The choice of developing a webchat service was guided by the Improving Justice in Child Contact project, a partnership project across five European countries hosted by the University of Edinburgh, where the young expert group indicated that a webchat service felt safer for young people to use.
If you have an interest in supporting children and young people affected by domestic abuse, you might like to read our recent case study on our work with the Improving Justice in Child Contact project, involving a partnership of women’s rights organisations from five countries.
Children and young people’s input has also shaped Scottish Women’s Aid’s understanding that children and young people impacted by domestic abuse want to speak with someone who ‘gets it’ – that is, who is specifically informed by their understanding of domestic abuse.
The CYP Webchat pilot was an additional service added to the Scottish Domestic Abuse and Forced Marriage Helpline (SDAFMH) from 1 December 2020 to 31 March 2021. It was one in a parcel of developments funded by the National Emergencies Trust as a Covid-19 emergency response and ran as a pilot project to explore this type of support and how it might be delivered.
Our Highlights Report draws together evaluative evidence gathered by Scottish Women’s Aid between November 2020 and April 2021, highlighting key points of learning and impact.
This report is organised by three different processes or activities undertaken by the pilot project:
Jump straight to the report now
Consultation with children and young people provided valuable insights into the perspectives of children and young people, e.g. around the language to use when speaking about domestic abuse. The design of the service and the publicity material was found to reflect this input.
While children and young people’s use of the service was limited during the pilot period, coinciding with Covid restrictions, the project yielded valuable learning around the service offer and practice learning on how to run a webchat service well.
Children and young people’s involvement in the service design and the conversations within Scottish Women’s Aid and between the organisation and local groups/wider networks, emerged as strengths in the process.
The project will lead to the bringing together of a resource pack for local Women’s Aid groups that wish to further develop their services to children and young people by offering webchat support.
Generated as part of the Covid response, this work tells the story of how the project was impacted by, and had to show agility in navigating, the constraints of the pandemic, particularly school closures.
This work highlights how moving through the structure provided by our approach can help you to surface and generate all of the learning you do through a short-term and creative piece of work like this, with a light-touch, pragmatic and flexible approach to gathering evidence against the project’s outcome map.
If you are interested in capturing learning and impact around young people’s participation in programme design, you might like to read our report on our work with the Life Changes Trust, and watch their film ‘Side by Side’.
There is a great fit between the values and principles of participation and co-production and our approach to outcome mapping and evaluation. If you are working in this area and would like to have a conversation to learn more, please do get in touch.
Alternatively, you can sign up to our mailing list if you would like to hear about further developments in our work in this area.
When used well, quotations have the power to give an immediate and unique insight into the perspective of the person who has offered it. This brings many benefits.
Using quotations well:
Your organisation may have good reflective practices already in place, and it’s likely that your relationships with the people that use your services offer rich opportunities for gathering voice. But how do you use these assets to their fullest potential to learn, improve and tell the story of how your activities make a difference?
Building skills and confidence in qualitative analysis, and implementing consistent recording practices and processes are important first steps that will help you draw together this kind of information easily when it is needed.
Your organisation may already use voice powerfully and effectively to communicate your work or to advocate for and with the people you serve, for example in case studies or funding bids. At Matter of Focus we advocate for building on this practice as part of a structured evidence base, which, in combination with other forms of evidence and over time, creates a clear “contribution story”.
A contribution story is an evidence-informed narrative that connects an organisation or iniative’s activities (“what we do”) over a series of steps to the outcomes or impacts they are seeking to influence (“what difference this makes”).
When we work with initiatives to help them understand and build their contribution story we use a process called outcome mapping that takes a plain language approach to understanding change, using our headings. For more on this see our post: A simple framework to help you understand change.
Using voice well to support your evaluation involves a series of steps:
If you have already created an outcome map and pathways for your initiative, these will help guide you in working out whose voices you need to include and provide a focus for data gathering.
It is important to think about who will notice the differences or impacts you are seeking, and how they will know. (What will they see? What will they hear?) This could range from a family member who notices a change in a person using your service to a senior decision-maker who notes a policy or system change. Often, people realise the importance of collecting the experience of people who benefit from initiatives, but neglect the insight and reflections of colleagues or other people who might be important to understanding change.
When gathering data in our work at Matter of Focus, we often match the questions we ask people to our six outcome mapping headings and to the particular project, programme or organisational outcomes that sit under each of these:
This helps to structure reflections and pull data together once collected.
There are many opportunities for recording voice as part of everyday practice. You may find that you are doing some of these already or are able to adapt to them very easily.
It’s important to have a system to keep track of the data and evidence you’re collecting. This post looks at how OutNav streamlines your data and evidence into one system.
Collecting data from or about people falls within the rules, regulations and ethics of the setting where it is collected. If you are gathering someone’s thoughts or experiences for one purpose, for example for a support plan, or when discussing an individual case with a partner agency, it may not be appropriate to use these publicly. It is good to be very clear with people about how and why you want to use their data, and to get their permission for this when you collect it. Whether or not you need permission will also depend on your organisation and data standards – consider anonymity, context and who will see the data.
A spirit of openness and respect is key to ensuring a good process for everyone involved.
Before reporting or presenting quotes you need a process of critical analysis and interpretation. This involves reviewing the data and selecting quotations that illustrate what you think is important. It is particularly useful to go through the analysis and interpretation process with other people to reach collective understanding. OutNav provides a clear and versatile structure to help with this process – see our post on how this works: Track your progress and impact in OutNav.
A process of analysis usually involves highlighting common themes whilst also being aware that including minority views can be important – if one person’s experience is very different from the rest what can you learn from this? Your analysis will be stronger if you can connect the dots between evidence that comes from different sources, for example if there is alignment between what people say and what you or someone else has observed. As part of your critical thinking and sense-making, it’s important to reflect on what might have influenced the voices you have heard, for example do the people you have spoken to dislike being interviewed, and are people agreeing because they want to please you or don’t want to lose the service?
Presenting your work well – in reports, case studies or other presentations, makes a huge difference to how voice and your contribution story will be heard. Based on our experience, we offer the following top tips.
“For a recent evaluation, I used a SmartSurvey to find out how young people experienced receiving individual grants. I included questions that combined a multiple-choice response with an open-ended follow-up asking ‘why?’ I presented the quantitative and qualitative data side by side in a simple table so the reader can see how many people responded in a particular way, alongside sample quotations to give a picture of the meanings behind this. In choosing the quotations, I was mindful to pick a small selection illustrating the main kinds of response – in this case more general statements of appreciation and very particular impacts on education and employment the funding had brought”
Watch our recorded live webinar: Getting going with qualitative data and analysis