The seven key principles agreed at Barcelona’s European Summit on Measurement are set to consign AVEs to history, reports Kate Magee.

In June this year, the first ever global standard for measurement was created at the second European Summit on Measurement in Barcelona.

The leaders of five global professional measurement and evaluation bodies – AMEC, Global Alliance, IPR Measurement Commission, PRSA and ICCO – and 190 delegates from the world’s top PR agencies and measurement firms agreed to seven key principles. The wording was finalised a couple of weeks ago, with the declaration now published in its final form (see right).

The principles include advice such as: clip counts and general impressions on their own should be avoided as they are usually meaningless; campaign outcomes need to be aligned to business objectives, and the effect on sales should be considered for consumer or brand marketing.

Despite being an important step forward for a discipline that is vital to proving the value of PR, the principles were criticised by many in the PR industry for being too pedestrian. Why did the declaration not deliver a final blow to Advertising Value Equivalents (AVEs), which are so widely scorned by practitioners, and what does ‘social media can and should be measured’ actually add to the debate?

AMEC executive director Barry Leggetter, who organised the conference, defends the principles, calling them an important framework for future development and a good way of starting a debate about standards of measurement: ‘Programme measurement has always been the Cinderella of the PR industry, coming behind creativity. PROs have always said “must we talk about that?”. But there are terrific conversation threads going on at the moment. There’s such a momentum in the industry that this is our chance to make a real change.

AMEC has more than 90 members in 33 countries. Some markets are at a more advanced stage than others and we need to make sure no-one is left behind,’ he says.

Leggetter says AMEC’s next focus is to validate new metrics to replace AVEs. ‘It doesn’t matter who you speak to, people use AVE because it is an easy thing to figure out,’ he says. ‘The metric is flawed, but it provides a number. That’s what a CMO or CEO demands, a number to show the PR programme is working.

‘It’s no good the industry saying the measure doesn’t work. We need to take responsibility to replace it. We’re very bullish about that and we will make it happen in the next six months,’ he says.

AMEC has the backing of both the CIPR and the PRCA in the UK. The PRCA is in the early stages of working with AMEC to produce a practical guide on meaningful evaluation for its members.

‘Everyone says they want practical advice. Most guides are unhelpful because they are too theoretical,’ says Francis Ingham, the PRCA’s director-general. ‘This guide will have different levels, explaining everything from DIY evaluation to outsourced evaluation with bells and whistles,’ he says. The PRCA will then include a section on evaluation in its awards scheme and will add a module to its consultancy management standard (CMS) accreditation to mark consultancies on their evaluation practices.

AMEC is also planning to contact wider PR award schemes to encourage a change in criteria. ‘At the moment, AVEs are still used to measure a campaign. We do not want to see AVEs as the sole measure,’ says Leggetter.

The other major focus for AMEC is to clarify best practice for social media measurement. No-one has a definitive solution for the constantly evolving platform. But the CIPR is trying to tackle the problem by upgrading the social media advice it released 18 months ago.

The CIPR set up a social media measurement group in June, inviting people to give their opinions, and will pull together a final document by the end of the year. It will also submit advice to the CIPR’s updated measurement and evaluation guidelines, due out in September. Both social media and AVE replacements will be discussed at the next key event, the North American measurement summit in New Hampshire on 6-8 October 2010.

- What should replace AVEs?

Speak to most PROs and they will rail about the inadequacies of the AVE measure. And yet, according to a PRCA survey conducted in May, 37 per cent of the 180 agency-head respondents admitted to still using the evaluation model. AVE is a simple, cheap number to calculate, clients ask for it and understand it, and there is not an easy alternative. But as Report International’s director Mike Daniels argued at the AMEC conference, ‘PROs need to move away from measurement as a media analysis tool to something more, as business decision support.’

The alternatives include econometric evaluation such as the COI’s recently introduced cost per impact measure (see box overleaf). Market-mix modelling, which allows clients to compare sales figures with outputs from different marketing disciplines, is also a popular choice.

- What is the best alternative to AVE?

True outcome measurement based on what motivates people to do business with your organisation or donate, or vote, or whatever your mission is,’ says Katie Delahaye Paine, CEO, KDPaine & Partners.

‘PROs need senior management to define what they mean by “success”. I call it the “Kick Butt Index” – if the boss says “you really kicked butt” what do they mean, and if they say “we got our butt kicked” what does that mean? That’s a far more meaningful metric than a made-up one such as AVE,’ she says.

Spectrum Consulting MD Mark Westaby argues that the PR industry should be using insight, not AVE, to measure effectiveness. ‘We recently did some work for a well-known restaurant chain that wanted to evaluate its online and social media,’

he says. ‘An AVE approach would have told it how much coverage it achieved, but our approach revealed not only that it should be using online reviews to generate more sales, but also the exact restaurant locations where these would be of most benefit.’

- Social Media How should PROs measure them?

The real answer is that no-one has a definitive answer. The constantly evolving platform demands a constantly evolving method. But there are broad points PROs should consider.

Philip Sheldrake, chair of the CIPR’s new social media measurement group and founder of Influence Crowd, says: ‘The principle that most applies to social media measurement is a focus on outcomes, not outputs. Getting obsessed over how many followers you have on Twitter and Facebook, or how many “likes” you got is a waste of time. That is measuring because you can, not because you should. Measures such as social mentions and social clout are absolute rubbish.’

He adds: ‘It is increasingly apparent that there isn’t a set group of people or “influencers” that you need to have a relationship with. A decision to buy something is a sedimentation process built over many years. It is not taken because five “influencers” told you to do it. This is disappointing for many PROs because we’ve left behind the simple world when PR boiled down to media relations. Social media have returned PR to its role of reputation management. It’s not longer a simple world.’

Spectrum Consulting director Mark Westaby fears that, because it is not a simple process, the PR industry will fail to take the lead on social media. ‘The PR industry never seems willing to take the steps necessary to address what are pretty complex communication issues. Unless it’s simple to understand, PROs don’t bother,’ he says.

He agrees with Sheldrake that the key does not lie in the number of followers on a social network. ‘Too many PROs believe that dealing with social media is down to how you handle bloggers, or setting up a Facebook fan page, but it’s not called social media for nothing,’ he says. ‘What really matters isn’t whether a blogger will talk to you, it’s understanding how members of a social network interact, how important emotion is in driving social media conversations, or getting the balance right between listening and driving conversation.’

Case Study COI

At the end of last year, the COI introduced radical changes to the way government PR is evaluated. This included the removal of AVEs from the core set of metrics and the introduction of a new measure to calculate ROI: cost per impact. This method divides the amount spent on PR by ‘impact’, calculated by multiplying reach by opportunity to see.

The COI’s PR and client director Oliver Hickson, who attended the Barcelona conference, says this measure is used by other marketing disciplines, so it allows the effectiveness of PR to be measured alongside advertising or direct marketing: ‘This is a more sophisticated way of looking at measurement. It’s really advantageous to use the same metrics as other marketing disciplines. It allows PR to be considered in the same way, rather than seen as separate, and shows PR is robust.’

This process included a huge programme to educate government departments about the new metrics. Hickson ran a series of workshops explaining the basics and urging people to decide on clear objectives before commencing work. ‘It was very well received by both central and local government. There is a real appetite to learn how you evaluate work properly,’ he says.

Hickson believes the most pleasing thing to have emerged from the Barcelona principles is the commitment to measuring outcomes as well as outputs.

HOW I SEE IT

Simon Warr, Board director, comms and public affairs, Jaguar Land Rover

In the absence of anything that is more relevant, we do use AVE.

Some measurement companies have their own ways of coming up with numbers, but these are often abstract and difficult to make meaningful. As long as your methodology is consistent, you can track one investment over another over a period of time with AVE. Internally, they have a degree of recognition and are something people can easily understand.

In my experience, however, senior executives are more swayed by pure coverage. If they see coverage of a brand, product or business in a media brand they respect, they think that’s good. They tend to look at share of voice in comparison to the competition and sentiment, rather than a financial measure. AVE is a completely made-up metric because you haven’t saved the firm money by not advertising. I’ve never experienced an advertising or marketing budget being cut because PR can get the media space more cheaply.

The holy grail for the brand and product space is to really track activity to the sales funnel, but even this is difficult to measure.

We have started to track coverage to dealer activity, but we are not particularly effective at this – partly owing to the scale of the business and partly because PR is such a small proportion of the overall business, the project often gets deprioritised.

We are increasingly launching dovetailed marketing activity, but we can’t isolate the influence of individual disciplines. And even if we could, a consumer may have seen an advertising campaign three months ago, then spoken to their brother-in-law, and then decided to buy a new car. How do you isolate the influence on an individual?