There’s been a swell of activity around outcome measurements and impact assessments across the youth sector, from new initiatives to help the youth sector improve its collective measurement to social investment bonds, which pay services handsomely, but only if you can demonstrate measure outcomes. Outcomes are ubiquitous and omnipresent.
Now, don’t get me wrong as I launch into an alternative to outcomes. I’m not even a ‘convert’ to outcomes; my first job was at a national statistical agency, I’m dyed in the wool numbers. And I think they pose a much needed challenge to direct service providers. All too often I’ve heard so many dubious claims made by services, from ‘without our work here, most of these kids would end up in real trouble with the police’, ‘our kids are able to stay in school because of our once a month support’ to my favourite vague claim ever ‘we do early prevention’. If you’re saying stuff like that, soz, you got to prove it. I appreciate in an era of fake news and vaccines-cause-autism dialogue, the thought that you have to prove your claims may seem ‘old school’, but hell, call me a stickler for the idea of an empirical truth (if you’re making empirical claims).
The problem I think for many direct service providers is that they have spent too long alluding to a breadth of audacious outcomes for their young users that prove extremely challenging to document. There hasn’t been much clarity about what the ultimate purpose of a programme or piece of work is, which make it almost impossible to measure any outcomes or cost up impacts from the work. How many services have you heard that have said ‘but it’s just so hard to measure all of the huge variety outcomes we achieve for our young people, like that one young person who we helped get work experience and the other who we helped get a laptop’ or ‘there’s just no measures available for the broad type of work we do’? Lack of clarity about your direct service’s aims makes outcomes impossible.
And so far, this defence of vagueness has kind of suited us. Being honest, when was the last time a potential donor said to you ‘gosh, and without this amazing girls night I may donate £££ to, all these disadvantaged kids would end up costing the state more because they’d get in trouble’ and you stopped and said ‘actually, that may be true in some cases but not all, and it’s not the reason we do it. We do it so that they will have a safe space to be girls and have something free to do when it’s dark out’? Brave. Or foolhardy.
But the times they are a changing, and it seems like there are two valid responses available to us.
Firstly, we can embrace outcomes and work them. A number of organisations have done the extremely hard work of developing a theory of change (and I don’t mean the paper exercise variety) that has helped them refocus and direct their work. If you’re clear about what the ultimate purpose of your work is, and you focus in on this, then measuring your success is the easy part. (E.g. if your purpose is to run a programme that improves the self-esteem of individual young participants, you can measure that. You just then can’t also claim your programme reduces recidivism among young offenders, unless you’ve measures that too or there’s some research that links esteem to reoffending rates). Clarity is key, but I’m not going to talk about that, because there’s so many awesome resources out there to help.
What I will propose is an alternative that I’m musing over; a rights-based retort. A rights-based approach would focus the aim of a service as a matter of entitlement for the young, without the need to justify itself through claims to generate a particular type of changes in service users.
Outcomes frameworks (for direct services) often imply that the purpose of the work is to generate some sort of future-oriented change in the individual young persons who use a service. I say future-oriented because, ideally, that change would be something that research suggests holds in to later life and produces a range of other positive benefits (aka ‘early intervention’). This could be something like ‘improve wellbeing’ or ‘reduce risk taking’. But often, I think services aren’t trying to do this at all. I think the reason many have trouble clearly describing the changes they want to see in their service users, the reason the ‘measures are all wrong’, is that services aren’t aiming to change young people at all. Or at least, aren’t aiming to change them in ways that have already been deemed desirable through theory of change exercises. Its possible they’re actually just not early intervention services. Scary to think about in this funding climate, but here’s a conceptual alternative.
Young people have human rights, which mean that without reason or conditionality, they are entitled to certain things. These things include:
- The right to play
- The right to safety, including safe spaces
- The right to express themselves
- The right to health (including sexual health and mental health)
- The right to be heard on issues affecting them…
…and so on. I think it’s possible that for many youth services, a better description of their aim is to help young people realise particular rights. For example, the girls group above could be described as a service designed to realise young women’s right to play and be safe.
Now you could get all techie and say you could measure outcomes from this and other rights-based aims, which I agree with. If you’re saying your offer is realising these rights, I’d probably want to see you gather some evidence that the young people you’re working with also see it that way. And yes, using pre and post measures and controls could be an awesome, convincing way of doing this. (Maybe that’s the middle ground that will help sell it in the current funding climate; your outcomes could be the measurement of rights realised).
But I see this as a substantively different activity to measuring an ‘outcome’ as a change you want to see in a young person because they have used your service. This is measuring a change in the opportunities and capacities available to that young person because of your service. This is moving from an outcomes measure that focuses on changes to individual young persons, to an ‘outcome measure’ that focuses on changes to young people as a collective. It’s a human rights and human capacity based approach.
Either way, there may be something in the idea of moving from an individualised outcomes framework to a collective rights based rationality. It may give us a better way to describe our services without having to imply that every young person we work with is a juvenile delinquent in waiting, an implausibly young mum/dad in the making, or a mental health crash about to happen… our services could just be there to help realise the rights our service users have, without condition or expectation of later disaster. All young people are entitled to services, regardless of whether or not these fundamentally change who they are to become.
Is it fundable, though? Who knows. Could this be used as another way to conceal poor services? Probably. But it’s an idea that could be worth thinking about.