Evaluation design - using action research principles to tailor evaluation to what practitioners want

We are pleased to welcome our guest blogger Eliza Buckley from IVAR, who is leading the NYSAF evaluation process.

Pears Foundation - 25 July 2017

Last week, we had the great pleasure of hosting the latest in our series of learning events for projects funded through the National Youth Social Action Fund, facilitated by our friends at the Institute of Voluntary Action Research.
To explain more about the evaluation process we are taking with the National Youth Social Action Fund, we’re very pleased to welcome Eliza Buckley from IVAR as our latest guest blogger.  We asked Eliza to outline the approach they have taken, but also to reflect on what the experience has been like.

 

Since September 2016 I’ve been working with colleagues at IVAR on an evaluation of the National Youth Social Action Fund (NYSAF) which is supporting nine projects across the UK, each delivering work to support young people from lower socio-economic groups or rural areas into social action – activity such as volunteering, fundraising or campaigning. 

The Cabinet Office and Pears Foundation (who fund NYSAF) commissioned this evaluation as an opportunity to use evaluation creatively – to ensure the process was just as beneficial as any report on outcomes. Working with the funders and nine grantees, we designed an evaluation that focuses on the issues practitioners say will help them most. In this blog, I want to describe how we’ve designed and supported the process that got them there. 

We began in December last year, at a day-long workshop with grantees, funders and partners in the Churchill Room, Whitehall (cue break for selfie on the balcony from which Churchill’s victory speech was delivered!). We worked hard and covered a lot of ground to arrive at two important things:

1. A shared understanding about what NYSAF means to us as a group:

- Demonstrating and raising awareness of the diversity of activity that exists in the field of youth social action

- Giving all young people the opportunity to engage in social action

- Doing more partnership working – developing more ‘joined up’ approaches

- Learning together about young people’s social action journeys – especially what happens before and after

2. A deeper understanding of the key challenges and opportunities that youth social action is facing and how the evaluation might help. We used ‘open space’ to allow these issues to emerge, make space for discussion, and give permission for themes to be left to one side if they ran out of steam.

Since December, we have worked with the projects and funders to explore what people felt were the bigger, tricky issues that the evaluation could help with:

Partnership and pathways – what can we learn to support young people into social action.

NYSAF projects are operating in a context of increasing competition for diminishing resources. In light of this, we have heard about a reluctance or lack of ability to collaborate, challenges around gatekeeping and reductions in local infrastructure removing previously well-established and reliable routes into social action for young people.

Community impact – what does this look like, what helps achieve it, what are the benefits of achieving it?

The benefits of social action for young people and volunteering journeys are relatively well understood. This is less the case for the ‘other’ side of the two-way benefit – the benefit of youth social action on communities. NYSAF participants thought that being able to demonstrate community benefit could aide engagement but also that the process of understanding this could be a tool for engagement itself.

We all came together again in April to work out what each project could collect or contribute in terms of data to support these themes and are currently in the midst of this activity. In July, we  me for a participatory analysis day to review what has been collected, what we’ve learned about the two themes, any gaps that remain and how we might tackle them together over the summer. Later in the year, we will begin thinking together about how to share the learning from this process with others, working closely with the #iwill campaign, a key partner in the research.  

It has been a fascinating, stimulating, challenging and rewarding journey so far – the projects and funders have entrusted us with guiding them through this process and, we hope, benefited from this along the way. For us as action researchers, this work enables us to live our principles, an excellent opportunity but one which requires regular conversations and alertness to note the changes that are happening while recognising that we ourselves are causing some of those changes – and, of course, all the while the environment around us constantly shifts, ahem general election anyone!?