How Should We Measure The Real Impact Of Culture?

The Old Vic is one of London’s most historic theatres, with a rich artistic heritage stretching back over 190 years. Actors such as Laurence Olivier, John Gielgud and Judi Dench began their careers here, and we provided the first home for the companies that would go on to become the National Theatre, the Royal Ballet and English National Opera. Since 2003, we’ve run Old Vic New Voices, the outreach and involvement arm of our work, which nurtures emerging talent, educates young people and promotes community participation.

We’ve been delivering this award-winning work for over a decade now, and have witnessed first-hand the real difference that our projects make. We see participants gain new skills, make life-long friends, transform their confidence and enhance their health and wellbeing. We see these changes, and we read people’s positive feedback, and yet like many other cultural institutions, we have a hard time converting this into ‘hard’ evidence about the impact of Old Vic New Voices. We can offer our funders compelling case studies and personal stories, but would love to reinforce this anecdotal evidence with the corresponding data.  Furthermore, better analysis would enable us to assess our work even more comprehensively, and to add even greater value for participants.

So why do we find it challenging to collect and evaluate this information? On one level, we’re always pushed for the time and resources needed to implement more rigorous assessments. But more significantly, we struggle to find a suitable evaluative model. Our previous impact assessments, some of which have been produced by external evaluators, have primarily been conducted via online surveys and participant interviews at the end of projects. These have produced useful data about the short-term effects of our projects for individual participants.  However, because both the personal and societal outcomes of creative projects are incredibly varied and multifaceted, we require a method that can capture this complexity.

We’ve used frameworks like the New Economics Foundation’s ‘Five Ways to Wellbeing’, which provides evidence-based actions for improving happiness and wellbeing. Other organisations have had success with the NEF’s Five Ways too, notably dance house Sadler’s Wells, who used them as the basis of a comprehensive project evaluation completed in association with Goldsmiths’ Institute of Cultural and Creative Entrepreneurship. Resources such as this are incredibly valuable in establishing a shared language, but they don’t always result in the kind of data that can be rigorously evaluated.

In addition to a lack of appropriate evaluative frameworks, we find ourselves facing broader ideological questions too. There is an understandable reticence within the cultural and creative industries when it comes to proving the social value of art. For many, the arts have an intrinsic worth that cannot be mapped or measured, with the primary benefit found in creative self-expression itself, rather than the longer-term effects experienced afterwards. Unfortunately, this is at odds with a funding landscape increasingly dominated by cost-benefit analysis. Should the output of our cultural institutions be exclusively focused on adding maximum social value? Or should we empower artists to create work they believe in, and trust that audiences will take something positive from it? It’s an increasingly contentious question, and one which is currently being considered by the Cultural Value Project. Established by the Arts and Humanities Council, the project is seeking to “make a major contribution to how we think about the value of arts and culture to individuals and to society,” in order to establish how best to measure that value.

We are therefore keen to find an evaluation method that can take into account the broad range of outcomes our projects produce, whilst also ensuring that artistic impact is not superseded by economic value. The answer to this may be found in one of fundraising’s favourite buzzwords: storytelling.

Storytelling evaluations, which take first person narratives as their basis, don’t try and fit people into a box. By using people’s own words, this approach can represent complex experiences and outcomes. Participants are then encouraged to self-analysize what the story relates to, positioning their account within a pre-established framework of say, health, happiness or social relationships. These stories, and where they are placed within the framework can then be digitally collated and analysed, in order to provide a valuable insight into what has been a success and why. And with participants self-reporting online, this approach need not be too costly or resource heavy. The Global Giving Storytelling Network has done innovative work in this field, using this method to measure the impact that various donations have made in communities in rural Kenya and Uganda.

At The Old Vic we’re excited about the possibility of applying a similar approach to arts and culture. Storytelling is at the heart of great theatre, and may also prove a valuable tool for us to have an even greater impact in our community.

Many thanks to Natasha for her insights and commentary on the state of impact measurement within the arts and culture space. Be sure to follow her on Twitter, and of course, both The Old Vic and their outreach programme, The Old Vic New Voices.

To stay up to date with the latest Markets For Good articles and news, sign up to our newsletter here. Make sure that you are also following us on Twitter.