It’s England, Jim, but not as you know it

If I were to name my gurus, then high up on that list would be Ben Goldacre. He’s the writer, broadcaster and doctor who made his name by debunking the myths that have arisen from flawed research studies and misleading analysis of scientific data. In his best-seller ‘Bad Science’ he unpicks the myriad ways in which the press (and others) undermine and distort science in their quest for headline grabbing stories, and explains how the manipulation of data can persuade clever people to believe stupid things, in some cases to devastating effect.

I wonder what Ben would make of Arts Council England’s (ACE) latest report, 'This England: how Arts Council England uses its investment to shape a national cultural ecology', which forms the basis of its submission to the forthcoming Culture Media and Sport Select Committee enquiry. ‘This England’ sets out to “restate the principles on which Arts Council England invests in England’s arts and culture, with underlying principles, guiding metrics and updated analyses”. In other words, it aims to explain ACE’s investment strategy and provide the supporting evidence to back it up. So far so good. But with Ben’s warnings ringing in my ears, I took ACE at its word about being ‘open and accessible’ about the report (p38), requested the actual numbers behind the ‘This England’ analysis and set about a page by page examination of the evidence. These are the highlights.

modest levels of investment in the regions have led to modest growth in engagement outside London; the £billions spent in the capital itself have resulted in fewer people engaging with the arts

Is ACE the country’s largest arts funder?
Does size matter? Well, it appears to matter to ACE. Someone has gone to the trouble of doing a bit of what Ben calls “cherry picking”, selecting data for a pie chart that proudly announces on page 8: “Arts Council England is the nation’s public investor in arts and culture, although we are not the largest funder”. Given the use of the present tense, you might expect this chart to illustrate the most recent figures (2012/13), which ACE tells me are £440m ACE Grant in Aid (but for some reason this figure excludes museums and music education hubs), £317m ACE Lottery, £547m direct DCMS funding, and £628m from local authorities. These figures make it only too clear that ACE is England’s largest single funder of arts and culture. But for reasons best known to itself, ACE has chosen instead to present a chart showing historic average funding levels dating back to 2005/6. They tell the opposite story, that ACE is not England’s largest single funder of arts and culture. Curious, to put it mildly.

The value of Strategic Touring funds
When reporting research findings it's good practice to quote actual numbers – “concrete numbers, just like the ones you use every day to check if you’ve lost a kid on a coach trip or got the right change in a shop.” Ben reckons – and I agree – that percentages, or anything even slightly technical or processed, won’t necessarily tell the same story. In this case, the bar chart of percentages on page 10 reveals that “While organisations with London postcodes received 42% of the financial value of [ACE’s] strategic touring programme, only 8% of the value was delivered there.” But let’s take another look at this in the context of the amount of money involved – a total of £24m over two years. In ‘concrete numbers’, during that period ACE’s Strategic Touring programme distributed just over £10m to London organisations, while Londoners actually benefited from this fund to the tune of just under £2m. That’s roughly £8m, or £4m a year, leaving London. Why not tell it how it is? Because in the context of ACE annual expenditure totalling £760m it’s not such an exciting story. That’s why.

Who is engaged?
Charts and diagrams typically flag up what a report’s author really wants us to pay attention to and This England is no different. But determined not to have the wool pulled over my eyes, I started to unpick some of the numbers that aren’t represented in the pictures. Based on the Government’s ‘Taking Part’ statistics, we are told on page 11 that  “between 2005/6 and 2012/13, the highest rates of increase in arts engagement were among people living in the North West, Yorkshire and Humberside, East Midlands and East of England, historically places of lower engagement. This is a major achievement.” Why no diagram to illustrate such a triumph? Well, probably because the very same analysis that delivers this conclusion reveals – if you choose to reveal it – that engagement in the arts by Londoners actually fell during this period. This is potentially dangerous stuff. One might, for example, draw the conclusion that whereas modest levels of investment in the regions have led to modest growth in engagement outside London, the £billions spent in the capital itself have resulted in fewer people engaging with the arts. Hmmm…

What’s a core city?
Next up on page 11 is a map showing investment in England’s ‘core cities’, although what constitutes a ‘core city’ is something of a mystery. Included is little ol’ Cambridge, with a population of around 134k, but bad luck Southampton (237k), Leicester (330k), Brighton (273k), Hull (256k) – the list goes on – you’re not core, apparently. ACE believes its pretty map “shows how our investment is spread geographically if you compare spend per head by core cities, rather than by government region”. But what baffles me is how this is meant to help anyone understand the efficacy of ACE’s investment strategy – especially when the concept of a ‘core city’ is neither defined nor even gets a mention in its 10-year strategic framework ‘Great art and culture for everyone’ (you know, the one that this report is supposedly providing the evidence to justify). Apparently, the figures also reflect ACE’s “strategy of clustering investment in major urban centres”, although mention of this strategy is also notably absent from the aforementioned ‘Great art and culture for everyone’.

But, setting aside the purpose of these figures for one moment, it’s worth looking at the analysis behind the map showing National Portfolio investment in core cities and surrounding areas. In the small print we read that this excludes investment in most – but not all – of England’s largest cultural organisations: English National Opera, Royal National Theatre, Royal Opera House, Southbank Centre, Birmingham Royal Ballet, Royal Shakespeare Company, Welsh National Opera, Opera North and Northern Ballet. We are told it is “common” to do this in ACE’s standard analysis and reports, but apparently not so common as to be applied to the figures on page 27, where three colourful pie charts illustrate how well ACE is doing at weaning its NPOs off public funding by stimulating earned income and fundraising.  No doubt it’s useful for ACE to have income from the big London institutions counted in those figures.

It remains unclear as to why English National Ballet isn’t on this list, or Baltic and Sage Gateshead. All three receive more NPO funding than Northern Ballet, and ENB is a certainly a ‘national’ organisation (the clue’s in the name). But what does become clear, after a bit of digging, is that the organisations excluded from this analysis consume around 38% of England’s entire NPO budget. So it seems that ACE’s “common” practice is to conduct its funding analysis based on less than two-thirds of its investment. ‘This England’? Clearly not the whole of England then.

How successful are London’s G4A applicants?
One of my favourite parts of Bad Science is the chapter on ‘Bad Stats’ where it says: “With very careful selection of numbers, in what some might consider to be a cynical and immoral manipulation of the facts for personal gain, you can sometimes make figures say anything you want.” So I thought I’d test this out and have a go at generating a few different interpretations of the figures relating to ‘Grants for the arts’(G4A) applications. ACE represents these in the chart on page 19 and draws the conclusion: “A lower proportion of Grants for the arts applications from London based applicants are successful, due to the high volume of applications/demand from applicants resident in that region.” But I have discovered that, by analysing the very same data in other ways, we could, for example, write the following different but still factually correct headlines:

REGIONS LOSE OUT AS SUCCESS RATES OF G4A APPLICATIONS FALL AROUND ENGLAND
London applicants clocked up 3 times as many successful G4A applications in 2012/13 compared with applicants from the rest of England – a figure which has grown consistently since 2010/11, when the ratio was only 2.2:1 in London’s favour.

Or how about:
4 OUT OF EVERY TEN G4A AWARDS WENT TO LONDON AND THE SOUTH EAST IN 2012/13
London and the South East have consistently headed the league table of successful G4A applicants since 2010/11, with 40% of successful applicants now coming from those regions.

Or even:
LONDON G4A BIDS ARE DOUBLE THE NATIONAL AVERAGE
Successful G4A bids are now more than twice as likely to come from London as from any other parts of England: 742 awards were made to Londoners in 2012/13, compared with an average of just 247 across the other eight regions.

You’re right Ben – cynical manipulation of the facts really can work – and no one would ever know unless they took the trouble to do their own analysis.

Who really benefits from NPO touring?
Having almost lost the will to live, battling through the smoke and mirrors, I determined to explore the question of ‘who benefits’ from touring. This is at the heart of ACE’s assertions that everyone gets a fair share of the funding spoils, meaning it’s a prime candidate for further scrutiny. The figures presented on page 21 show that “London-based National Portfolio organisations retain one of the smallest proportions of toured activity within their regional boundaries”. In plain English, this refers to how much of London-based NPOs’ work is toured to other parts of London. Given that London only measures about 45 miles from East to West, and 35 from North to South, it would be rather surprising if London-based NPOs did tour a lot of work in their own region.

I thought it might be rather more interesting to use the same figures to find out how much of the London’s touring activity leaves the capital. Sticking with ACE’s preferred percentage measures, it emerges that London NPOs export just less than half (48%) of their touring activity to the rest of England – soundly beaten into fourth place by Yorkshire, the North East and the West Midlands, whose exports to the rest of England amount to 62%, 61% and 52% of their NPOs’ respective touring activity. But when it comes to international exports, London really does lead the way, exporting 30% of its touring beyond England, followed by the South West (28%),  the North East (27%) and Yorkshire and the South East (25% each).

The chart on page 22, which presents concrete numbers, manages to accurately quantify the net export/import of touring activity, and London is clearly a net exporter of touring. At last – a really useful piece of analysis. This must surely be a good starting point for debate on the balance of cultural capital across England. Deeper analysis of these figures reveals that Londoners have relatively little exposure to the touring output of the rest of England’s NPOs – less than any region other than the North East. Is this a good thing? Is it healthy that our regional NPOs get so little exposure in the capital? Or is the touring output of London’s NPOs so superior that it deserves a much wider audience? Surely these are the sort of questions that should be at the heart of a debate about the regional balance of cultural capital.

I could go on, but I think you already get the picture. ‘This England’ has been constructed from figures that have been carefully chosen to disguise a London-centric funding bias and is based on selective analysis that presumably set out to discredit those calling for a re-assessment of the geographic balance of arts funding. Ben Goldacre quotes the American physicist Richard Feynman on his research studies: “the idea is to try and give all the information to help others to judge the value of your contribution; not just the information that leads to judgement in one particular direction or another.” In its flagrant disregard for this principle, rather than present a balanced picture of its investment in England’s arts and cultural sector, ACE is in effect making a case for the status quo. And that just isn’t good enough. Its role as England’s national arts funding body requires it to take a balanced view and there can be no justification for the sort of analytical trickery that is all too evident in its submission to the Culture Media and Sport Select Committee. I’ll be interested to hear what the MPs think of it all – and what you think of it too Ben, if you’re reading. Thanks for the inspiration.

Liz Hill is Editor of ArtsProfessional