Evaluation

We have a tried and tested approach to evaluation. We use Theory of Change to underpin all our evaluation work. This means we focus not just on what the impact is, but on how the impact is achieved. This method allows us to make good projects better but also take the learning from projects that have struggled.

We start all our evaluations by creating an evaluation framework. We do this in a workshop to gain an understanding of the outcomes the project is trying to achieve and any challenges or barriers the project has or is likely to face.
We incorporate a mixed methods approach to evaluation, designing user-friendly tools to collect both quantitative and qualitative data. As well as collecting robust data using validated scales where necessary, we also believe in the power of stories to tell us what a project has achieved.

We finish all our evaluations with a Learning to Action workshop. This ensures evaluation is not a static external thing that sits in an organisation, rather the recommendations can become embedded into the organisation or the project and positive action can result from the work.

To find out more about our work click here.

Case study

Using local assets to deliver Sport for Change

About the project

In May 2016 Shephard & Moyes Ltd were commissioned to evaluate the Place Based Funding initiative (PBF). This initiative was the first collaboration between Sport England and Comic Relief to develop projects delivering Sport for Change. The power of sport as a driver for social change is being increasingly understood and the partnership offered an opportunity to pilot, not just a collaboration but also a new way of working. PBF takes an area based approach. It devolves decisions about what is funded, where and how much is down to local areas. There are two areas in this pilot; Burngreave in Sheffield and Lakenham in Norwich.

In January 2017, two further projects were added to the Place Based Funding Evaluation. In addition to the pilot project, Comic Relief have a Communities and Sport for Change funding stream. Through this some place based approaches are receiving funding. To further add to the insight and understanding of the Place Based Model two of these projects are being added to the evaluation. These two areas are Tower Hamlets and Hastings.

What we did

We are supporting the four areas to self-evaluate their projects.  The process started with an evaluation workshop in each area which used Theory of Change to identify a set of outcomes and research questions for each project.  Through this we identified a set of common outcomes across all four projects, as well as some which are more locally specific.

We used these to develop a set of common tools to be used across all four areas.  These included surveys which will measure outcomes around physical activity levels, wellbeing and aspirations, plus a suite of qualitative methods including interviews, worker observation and story telling tools.

We delivered training to recipients of the Place Based Funding monies in order to develop an understanding of why evaluation is important and how to use the tools.

The projects have just started to deliver and data is being collected which will inform evaluation reports in summer 2017 and 2018.  The self-evaluation data collected by project partners will be supplemented by independent research carried out by the Shephard & Moyes Ltd team.

What we found

In March 2017 we produced a progress report which provided a baseline position for all four areas, and progress to date in terms of planning and commissioning activity.   Learning from this initial report included:

  • A two-year timescale for projects with an asset based focus does not leave much time for delivery
  • Inclusion of a Community Development Worker in projects has played a critical role in helping understand the local areas and bringing Community Development expertise into the County Sports Partnerships
  • The asset based approach has enabled people to be engaged in a conversation about their area, however there will be an ongoing challenge around managing expectations
  • Despite minimal direction and prescription from the funders, areas have designed very similar commissioning processes and are developing similar ideas around developing activities that encourage families and people of all ages to come together, socialise and take part in physical activity.

Areas would have benefited from more direction from the Funders