Taking an Outcomes Approach:

Evidencing the changes your project brings about

1. What is the purpose of this document?
1

This document outlines the key principles of an outcomes based approach to
project evaluation. It aims to provide information for Youth Musics funded
partners that will be of use when thinking about how you evidence what
changes your work brings about. While the focus is on outcomes, the activities
and resources that influence these outcomes should also be critically
considered. This is the only way that we will be able to learn from what works
well or not so well in our funded projects, and be able to share those lessons
with others working in the sector. This document however is about outcomes
and what we consider a useful approach to being able to capture them.

Our approach is based on that of the Charities Evaluation Services (CES)
2
. We
would recommend that you consult their website for further information on
their approach to evaluation (www.ces-vol.org.uk). You may also be
interested in their training programme. In writing this document we have also
drawn on the Big Lottery Funds outcomes approach, as laid out in their 2006
guidance document
3
(which is also based on that of the CES). We would
encourage partners to consult this more detailed report.

We consider this to be a working document that will be added to over time
as we and our partners develop more ways of working effectively with this
approach.

2. What does the document cover?
There are eight more sections in this document:

3. Why monitor and evaluate your work?
4. What are outcomes?
5. Youth Music as an outcomes funder
6. Taking an outcomes approach
7. Outcome indicators
8. Collecting information on outcomes

1
This document does not address the critical topics of codes of practice and ethical guidelines
to follow when carrying out research. As a start, you may want to consult the United Kingdom
Evaluation services guidelines http://www.evaluation.org.uk/resources/guidelines.aspx on
these topics.
2
For the purpose of this document we have in particular drawn on Practical Monitoring and
Evaluation: A guide for voluntary organsiations, (CES 2005)
3
Burns, S. & MacKeith (October 2006) Explaining the Difference your Project Makes: A BIG
guide to using an outcomes approach
2
9. Making sense of the information
10. Developing this document and related resources

Throughout the document we use an example project Making Sounds to
illustrate the principles we are discussing. This is not a real Youth Music project
and the examples should be taken as illustrative rather than definitive samples
of best practice.

3. Why monitor and evaluate your work?
Before we move on to talk about outcomes specifically, we thought it would
be useful to explain why Youth Music considers evaluation to be a core
component of our and our partners work. Effective evaluation helps you to
assess how well you are doing and be more effective in your practices.
Obviously delivering projects is everyones priority, but it is critical that you
allocate resources to evaluate your work. It will enable you to:

reflect on your practice (both during and after a project) and improve
the quality of projects you deliver
assess whether you are reaching your target participants or whether
you need to amend the project to do so
feel more confident that you are delivering an effective project
identify both intended and unintended outcomes of your work
make a strong case when making future funding applications, by
having evidence of your achievements and lessons learnt.

In addition, by submitting the evidence to Youth Music this will help us to:
develop a body of evidence of effective practice in the sector to be
made available to all stakeholders
advocate on behalf of the sector on the wide variety of outcomes
music-making projects can achieve
identify areas of need, where achieving outcomes may be more
challenging and require innovation or extra resources
evidence the impact of the work of Youth Music and its funded
partners.


4. What are outcomes?
Outcomes are the changes, benefits, learning and other effects that you can
attribute to your projects activities- theyre the difference your project makes
(CES, 2005, p.53). While these will probably focus on the children and young
people taking part in your music-making project, they may also relate to a
parent or carer, your staff or your organisation as a whole (see examples in
section 6.2 below).


5. Youth Music as an outcomes funder
Youth Music is clear about the changes we want to bring about through our
activities and those of the partners we fund. If you have applied to Youth
Music for funding under a particular programme then it should be clear what
that programme is trying to achieve. You will be expected to be able to
relate at least some of the intended outcomes of your project to those of the
programme. As part of the conditions of funding you will be expected to
report to us on your progress toward achieving those outcomes.

3

6. Taking an outcomes approach
When taking an outcomes approach to your project design it is helpful to
think about it on three levels (see Diagram 1 below):
1. Its overall aim
2. Its intended outcomes
3. Its activities

Diagram 1: Charities Evaluation Services Planning Triangle (adapted by Burns &
MacKeith, 2006 on behalf of BIG
4
)



6.1. The overall aim of your project
What is your project setting out to do? What is the overall change or
difference you want it to make? Fundamentally this explains why your project
exists.

Making Sounds
5
- Overall aim

Project description
A project for young people in contact with a Youth Offending Team (YOT) in London.
This is a 12 month project where young people will learn a vocal based skill- singing or
beatboxing. They will attend workshops and will be linked to a peer mentor from a
local youth arts club, who will also attend the workshops.

Overall Aim
To improve the life chances of young people in contact with a London-based YOT,
through music-making activity.



4
Burns, S. & MacKeith (October 2006) Explaining the Difference your Project Makes: A BIG
guide to using an outcomes approach, p.8
5
Please note: This is NOT an actual project and does not necessarily reflect a project design
that Youth Music would/has funded.

The overall
aim of your
project
The differences you intend to
make or the changes you aim to
bring about for your
beneficiaries, the community or
the environment
The main services and activities you plan to carry out what
those working on your project will actually do
Overall Aim
Intended
Outcomes
Activities
4
6.2. The intended outcomes of your project
Your intended outcomes are the changes that you are aiming to achieve
because of your project (i.e. what do you expect to change through your
project? What difference will it make, for example, for the young people
taking part?). The language you use to articulate these should involve words
that reflect change, for example to: increase, reduce, expand, enable
develop, improve, etc. (CES, p18).

Your intended outcomes should be linked to the activities you will do (i.e. why
are you doing the stated activities? What do you hope to achieve through
delivering them?) The changes that result from your activities are your
outcomes.

They should also be specific, measurable, achievable, realistic and timely
(SMART). Think about the time and resources you have available and the
type of activities you are providing, this will help you to devise outcomes (and
relevant indicators, see 7 below) that are SMART. They must relate closely to
the activities (specific), be counted or described thoroughly (measurable),
and be achievable and realistic within the time and resources dedicated to
the activities.

Making Sounds
4
- Intended Outcomes

By the end of the 12 month project:
To have increased the musical skills of participants.
To have improved participants knowledge of music-making opportunities in
their locality.
To have improved young peoples social and psychological well-being.
To have increased young peoples motivation to engage in education and
training.
To have developed a more positive attitude to young peoples music-making
among parents/carers.
To have increased the skills and knowledge of the music leaders in working
with young people in contact with a YOT.


6.3. The activities involved
What are you going to do that will bring about these changes? As mentioned
above, these are the activities you are going to deliver through your project.
While you use change language for your outcomes (e.g. an increase in
musical ability), you should use doing words for your activities (e.g. by
providing 12 workshops).

You may find it helpful to think about these within the CES Planning Triangle
structure (see diagram 1 for an adapted version of this). This allows you to
think about each level and how they relate to each other. Will the overall aim
of your project be met through achieving the intended outcomes? Will the
activities you have planned realistically enable you to bring about the
changes outlined in your intended outcomes? While some activities will help
deliver on more than one outcome, it is critical that each activity is linked to
at least one outcome, and that each outcome has at least one activity that
will ensure your project delivers on it.



5
Making Sounds
4
- Activities

To run weekly workshops for participants during school term time.
To employ a skilled music leader to deliver high quality music-making
activities.
To provide pastoral care to young people over the course of the project, with
the support of youth workers from a local arts centre.
To provide an information pack to young people and follow up with 1:1
sessions with the music leader on progression routes and further local
opportunities for music making.
To hold 2 performances to which parents/carers will be invited and to send
them an information pack every three months about the project and key
achievements of the group.
To hold regular meetings between relevant YOT staff, the music leader and
youth workers to discuss the projects progress and any issues arising.


Once you have looked at your project within this structure you should ask
yourself the following questions developed by BIG for their partners:

When you have filled in a triangle, check the following:
Have you used words of change in the top and middle and doing
words in the bottom of the triangle?
Does the middle of the triangle describe what the changes you intend
for your projects participants, their parents/carers, your staff etc?
Does the bottom of the triangle describe what those working on your
project are going to do?

Now you can look at the triangle as a whole to check that the plans for your
project are realistic:
Look at each of your intended outcomes in the middle level. Will the
activities you have listed at the bottom realistically help you to achieve
one or more of your intended outcomes?
Now look at the activities. Does each one link directly to one or more
of the intended outcomes? If not, why is the activity included? You
may want to consider whether the activity really is important. It may
well be that the activity will help you to achieve another change. If this
is the case you may want to include another outcome. It may help to
do this if you number the outcomes and put the number(s) against
each activity.
From Burns & Mac Keith, 2006 (p. 10)


7. Outcome indicators
Once you have decided on your intended outcomes you will need to identify
the indicators you will use to evidence your progress in achieving these
outcomes. What would you see, hear, or read about project participants that
would show you had made progress toward your outcome? What would
participants be doing differently? So, before your project starts you will need
to decide what information you want to collect to evidence you progress to
achieving your outcomes (outcome indicators) and how to collect the
information needed. As with setting your outcomes, it is important that youre
realistic about the number of indicators you set out to monitor and what kind
of information you will be able to collect (see section 8 below).

6
Making Sounds
4
- Outcome Indicators

Outcomes Possible outcome indicators
Increased musical skills

Young peoples own assessment of musical
skills

Music leaders rating of young peoples
musical skills

Improved knowledge of music-
making opportunities in the locality

Level of understanding of own strengths and
weaknesses

Whether participants can think of several ways
of finding out about different local music-
making opportunities.
Improved young peoples social
and psychological well-being

Whether participants feel:
able to express themselves in front of
others
able to make decisions that they feel
are good for them
listened to by other people
like what they say and do will make a
difference to their lives
and that they have:
been able to turn up to the workshops
on time
learned to work well with other people

Increased motivation to engage in
education and training

Level of perceived motivation to engage in
education/training

Attendance levels at education/training
A more positive attitude to young
peoples music-making among
parents/carers

Level of parent/carer attendance at
performances

Parents/carers views on the value of music
making for their young people and young
people more generally

Increased skills and knowledge of
the music leader in working with
young people in contact with a
YOT

Whether music leader feels well-equipped to
meet the needs of young people in contact
with a YOT

Whether music leader can identify a range of
sources of support upon which he/she can
draw to be able to effectively deliver the
programme of work with this group of young
people.


Some of the outcomes you set out to achieve may be difficult to measure
and count. The CES (2005, p 23) give the example of the outcome to change
young peoples attitudes about social issues, which cannot be measured
easily, so you would need to use indicators that assess the change
approximately (proxies). They suggest that using time keeping and
attendance levels, alongside other evidence may be an indication of an
increased sense of commitment and responsibility, which relates to the
intended outcome.
7


8. Collecting information on outcomes
As mentioned in 7 above, before starting your project you will need to think
about how you will collect the information needed to demonstrate the
progress you have made in achieving your outcomes. Maybe you already
collect certain kinds of information as part of your monitoring systems that you
could use? Maybe you could tweak the monitoring system you already use
by rewording or adding questions? However, you may find that you need to
find a new way of collecting the information. If you are looking at change
you will need to think about collecting information at two points in time
(at/towards the start and at the end of the project). It is beyond the scope of
this document to provide a detailed account of tools/methods that you may
want to use, as these will vary from project to project and will depend on the
outcomes chosen
6
. For example, information on the outcomes for young
people could include:
Observation of their level of engagement/skills
Self-assessment questionnaires
Outcome scales
Diaries they keep as part of the project
Notes kept by project leads
Interviews
Focus groups
Samples of work
Etc.

As with all elements of this process it is important that you are realistic about
what data collection you can do within the given resources.

Making Sounds
4
- Data collection tools for increased music skills outcome

Outcome indicator Data collection tool
Young peoples own assessment of
musical skills
Young people complete at the
beginning and again at the end of the
project a scale that asks them to;
Please mark your responses to the
following questions on the record, where
1 is low and 8 is high

How do you rate your musical ability at
the moment?
1 being that you dont feel you
have any musical ability, 8 being
that you think youre at the top of
your game!

Music leaders rating of young peoples
musical skills
Music leader completes a log at the end
of each workshop, noting progress made
by each participant.





6
Practical Monitoring and Evaluation: A guide for voluntary organsiations, (CES 2005) provide
some examples of data collection tools and some initial guidance on data analysis.
8
9. Making sense of the information
The way you make sense of the data you collect on your outcomes will
depend on the kind of information you have collected and the way in which
you have collected it. For example, you will need to use a different approach
to analysing numbers to that adopted if you have done interviews with young
people. As with the tools for collecting data, it is beyond the scope of this
document to provide an in-depth account of how to analyse the data you
collect
6
.



Making Sounds
4
- Making sense of the information for increased music skills
outcome*

Young people varied in the extent to which they felt their musical skills had
increased over the course of the project.
On average, participants reported a 4 point increase (on an 8 point scale) in
how they rated their musical skills.
None of the participants reported a decrease in their skills.
Four felt they had only improved by 1 point. However, the music leaders
observations noted in the session log show that three of these participants had
initially ranked themselves toward the top end of the scale. Once they began
to learn more about their chosen vocal skill their self-awareness had improved
and they realised that they had more to learn to be top of their game. For
the other participant in this group of four, both he and the music leader felt his
musical skills had improved little over the course of the project.
*This example only deals with reporting on the actual outcome and does not take account of,
the activities and resources that influence these outcomes which should also be critically
considered.


10. Developing this document and related resources
Data collection tools
If you have developed any tools (such as questionnaires for young people) to
collect information on the outcomes of your project, and would be happy to
share them with others working in the area we would greatly appreciate it if
you could send them through to us. We would like to develop a repository of
tools that could be accessed by funded partners.

Let us know what you think
As mentioned in section 1, we consider this to be a working document and
would greatly appreciate your feedback on how we can improve it. We also
plan to develop it further by drawing on examples from our partners work in
this area. Please forward any comments to research@youthmusic.org.uk

Other useful sources:
http://www.biglotteryfund.org.uk/er_eval_explaining_the_difference.pdf
Big Lottery Fund- Explaining the difference your project makes: A BIG guide to
using an outcomes approach
http://www.biglotteryfund.org.uk/er_eval_self_evaluation_uk.pdf
Big Lottery Fund- Self Evaluation: A handy guide to sources

www.ces-vol.org.uk
Charities Evaluation Services

http://www.evaluation.org.uk
United Kingdom Evaluation services