Taking up the evaluation challenge

NHS mental health services traditionally offer behavioural and talking therapies, and medications. In children’s services, we know that these conventional interventions, while successful for some, result in significant improvement for only around half of those who access Child and Adolescent Mental Health Services (CAMHS). This assessment is based on a collection of measures routinely used in NHS services, and looks at patient outcomes in terms of mood, symptoms and activities.

Engaging with culture – the benefits

Alongside this understanding is growing evidence that creativity and arts engagement has been associated with benefits to both individual and community health, from supporting key ingredients of wellbeing such as expression, self-esteem and belonging, to spreading public health messages. During the pandemic and in lockdown, we have seen how culture and the arts can variously help to keep us calm, entertained, and connected. Perhaps unsurprisingly, the largest study of UK adult mental health during lockdown has identified having a creative hobby as among the strongest protectors of mental health.

While the role of arts organisations has traditionally been beyond the therapy room, their effects are something many of us can relate to. We play music to alter or augment mood, engage in cultural activities as a means of understanding (storytelling), or have experienced the enchantment of theatre – which research tells us can have the effect of synchronising heart rates and breath. A current study exploring the mechanisms of group singing has to date identified 600 distinct effects, from the physiological to behavioural. These activities, which we have engaged in across the ages and around the world, carry real-world benefits that we are only just beginning to describe. 

Measuring impact

A major issue for decision-makers is that arts providers have not routinely collected the kinds of standardised outcomes measures used by the NHS to evaluate this impact. Their methods of gathering evidence are commonly inconsistent and often idiosyncratic – and therefore not easily understood or considered at scale.

The basis of the medical and other clinical professions is that our services and interventions are evidence-based. Such evaluation is critical to any inclusion in statutory services. In mental health services, this relies on a set of national outcome tools, which are validated and identified at the national level by Public Health England, and, in children’s services, by the Child Outcomes Research Consortium (CORC). In Hampshire, for example, arts-led services are being incorporated into NHS provision in The ICE Project. ‘Inspire, Create, Exchange’ is an ambitious, innovative arts for health programme resulting from a collaboration between Hampshire Cultural Trust and Hampshire Child and Adolescent Mental Health Service. Their starting point was a shared view that arts and cultural activities can have a positive impact on our psychological health and emotional wellbeing, but demonstrating the disparate impacts of creative methods on wellbeing is a challenge. 

Commissioners want to know what is associated with good outcomes, and other important information such as the cost of an intervention and whether or not it impacts on the use of other NHS services, such as GPs and A&E. However, while they are unable to base decisions on personal accounts, commissioners also want and need to know what works for whom – and ideally, why. Those stories are critical to furthering understanding.

A new framework

Since July 2019 this issue has been explored by a GM i-THRIVE Programme working party, hosted by Manchester University NHS Foundation Trust. The THRIVE Framework for system change is an integrated, person-centred and needs-led approach to delivering mental health services for children, young people and their families. The team has been holding a series of cross-sector workshops with  academics, clinicians, arts partners including Arts Council England and young people’s mental health charity 42nd Street, aiming to agree on a minimum set of outcomes that arts and health providers can use as they seek to demonstrate their impacts to the health sector and commissioners. 

The GM Youth Mental Health Arts & Culture Evaluation Kit that has emerged from this is a practical and easily administered process which can be completed, including data entry, in just a few minutes. It has been designed to be simple, validated, relevant and useful to organisations themselves, at the same time providing the information that commissioners need. It still contains some of the outcome measures routinely used in the NHS, but it excludes those measures that use overly clinical language or ask inappropriate questions for these settings.

As a minimum, the evaluation framework includes an Outcome Rating Scale, a simple session-by-session measure that assesses tangible areas of life functioning known to change as a result of a therapeutic intervention. But it also includes the Short Warwick-Edinburgh Mental Wellbeing Scale which is widely used in research and by the arts sector to explore subjective wellbeing, and key demographics that provide wider understanding about the context to such outcomes. 

Since July 2020, 42nd Street have been testing out the kit with young participants who access their services, and they are reporting success. Young people – who do not always welcome a host of questions – have been happy to participate in the process. 

From theory to practice

Starting this month, a series of four ‘proof of concept’ projects will begin in localities around Greater Manchester, and their impact, cost and attendance will be evaluated with this new kit. In Bolton, for example, young people will be supported by their CAMHS keyworker to attend Wellbeing Theatre. The keyworker will participate with them in this intervention, which has been developed in collaboration with two local cultural providers, Odd Arts Bolton Lads and Girls Club.

The project will be drama-based and help young people to express themselves creatively through group work and script-writing, with the week-long programme culminating in a performance. Sessions will include games and will raise topics that are affecting people. The hope is that the group will support self-awareness and reflection, help people to re-engage positively with their peers, and improve their ability to access education. 

All being well, this will be a vital step in the more widespread understanding of and recognition for the role that arts and health providers can play in supporting the mental health of young people. 

Dr Katherine Taylor is Senior Clinical Psychologist / Arts, Culture and Mental Health Programme Manager for GM i-THRIVE
 [email protected]
 @GMiTHRIVE | @communikatt