Measuring Public Relations – still sucks but getting better

There are two answers I know of and neither are satisfactory to someone who strictly demands numbers – it’s the same for a host of other business elements that are hard to measure: the value of culture, leadership, holidays, marketing, advertising, staff morale and turnover, and more.

Yet all contribute to the three factors critical to an organisation’s performance: retention, growth, reputation.

What ever technique we use to measure, it’s really important that everyone starts out with the same expectations.

Public Relations: the first measure

When you employ PR folk, you get in return the reputation you deserve. Where are you, currently, on this table?

It’s about credibility versus visibility (if you have high cred. and low vis. = undiscovered; etc.). It’s perception versus reality.

In an increasingly competitive and chaotic environment companies that are perceived as under-performers lose customers, shrink and ultimately die. We all need to strive to be ‘unparalleled’ to stay competitive.

And these days we don’t have time to mess with perceptions. The figures vary, but on one survey the average US company survives for 15 years; on another, a CEO should stick around for a little less than 5 years.

So, in short, you and your organisation may only be around for a while, so don’t waste time putting reputation building on the back-burner.

When I use this technique to sort my strategy, I use it across different stakeholder groups: do we have appropriate visibility with the public, key politicians, key customers, prospects, etc?

Public Relations: the second measure

This is about measuring against goals. The clearer the expectation, and the better the measuring, the more satisfying the outcome.

While measurement is hard, it’s not for want of trying. In fact, there’s an organisation devoted to it, loftily called the AMEC – the International Association for Measurement and Evaluation of Communication. It has the 7 principles called the Barcelona Principles, updated last year (2015). It’s a tough read so be patient. See below.

In short the Principles are about (as clearly as possible) measuring against defined goals.

AMEC has provided a ‘grab-bag’ of measurement techniques, demonstrating public relations can be measured to a point, and it’s getting better.

PRINCIPLE 1: Goal Setting and Measurement are Fundamental to Communication and Public Relations

• Measurement and evaluation against defined goals and SMART (i.e. specific, measurable, attainable, relevant and time-bound) objectives are fundamental to good communication and PR programs. They are critical to any communication program, whether it be a single campaign or an on-going effort where the results are incremental over time

• Goals can be quantitative or qualitative yet should address target audience and what about them the communication program is intended to affect, how much of a change represents success, and by when this result should be achieved. This should be based on a clear understanding of the target audience including their current positions and views, as well as the context in which the program is intended.

• Measurement evaluation and goal-setting should take a holistic approach, including representative traditional and social media; changes in awareness among key stakeholders, comprehension, attitude, and behavior as applicable; and impact on organizational results. It should be integrated and aligned across paid, earned, shared and owned channels where possible.

PRINCIPLE 2: Measuring Communication Outcomes is Recommended Versus Only Measuring Outputs

• Outcomes include shifts in awareness, comprehension, attitude, behaviour and advocacy related to purchase, donations, brand equity, corporate reputation, employee engagement, public policy, investment decisions, and other shifts in stakeholders regarding a company, NGO, government or other type of organization.

• Practices for measuring the effect on outcomes should be tailored to the objectives of the communication program

• Both quantitative and qualitative methods should be used as appropriate.

• Standard best practices in target audience research including sample design, question wording and order, and analysis should be applied.

PRINCIPLE 3: The Effect on Organizational Performance Can and Should Be Measured Where Possible

To measure results from communication for an organization, models that determine the effects of the quantity and quality of communication outputs on organizational metrics, while accounting for other variables, are a preferred choice.

Related points are:

• Demand for models to evaluate the impact on target audiences of PR and communication activities in an integrated environment is significant and increasing.

• Practitioners need to understand the value and implications of integrated marketing and communication models for accurate evaluation of each channel including PR. There exists an ongoing need to develop PR measures that can provide reliable input into integrated marketing and communication models.

• Survey research can also be used to isolate the change in purchasing, purchase preference, engagement, recommendation or attitude shift resulting from exposure to communication initiatives versus other channels.

PRINCIPLE 4: Measurement and Evaluation Require Both Qualitative and Quantitative Methods

Tracking surveys can do a good job in measuring quantitative change in outcomes. Qualitative methods can often add a needed dimension to better explain the quantitative, and may in some cases be preferable to measuring overall results.

Specific to the measurement of media resulting from public relations programs, overall clip counts and general impressions are generally meaningless. Instead, media measurement, whether in traditional or online channels, should account for:

• Impressions among the stakeholder or target audience
• Quality of the media coverage including, but not limited to: Tone; Credibility and Relevance of the Medium to the Stakeholder or Audience; Message Delivery; Inclusion of a 3rd party or company spokesperson; Prominence as Relevant to the Medium
• Quality can be negative, positive, or neutral; the assumption should never be made that the results of a communication or public relations campaign or program are always positive or successful. Good measurement and evaluation allows for the possibility of negative or poor results from a campaign or program.

PRINCIPLE 5: AVEs are Not the Value of Communication

Advertising Value Equivalents (AVEs) do not measure the value of PR and do not inform future activity; they measure the cost of media space or time and are rejected as a concept to value communication, media content, earned media, public relations, etc.

Where a comparison has to be made between the cost of space or time from earned versus paid media, validated metrics should be used, stated for what they are, and reflect:

• Negotiated advertising rates relevant to the client, as available;
• Quality of the coverage (see Principle 4), including negative results; and
• Physical space or time of the coverage related to the portion of the coverage that is relevant

Multipliers intended to reflect a greater media cost for earned versus paid media should never be applied unless proven to exist in the specific case. This also applies to “pass-along values.”

PRINCIPLE 6: Social Media Can and Should be Measured Consistently with Other Media Channels

(PW: AMEC needs to regularly update this, the most exciting area of measuring against goals)

• Organizations need clearly defined goals and outcomes for social media.
• Media content analysis should be supplemented by web and search analytics, sales and CRM data, survey data and other methods
• Evaluating quality and quantity is critical, just as it is with conventional media.
• Measurement must focus on engagement, “conversation” and “communities” not just “coverage” or vanity metrics such as “likes”.

PRINCIPLE 7: Measurement and Evaluation Should be Transparent, Consistent and Valid

All measurement should use valid methods and be reliable and replicable in the case of quantitative methods and trustworthy in the case of qualitative methods.

The concepts of integrity, honesty, openness and ethics are critical to this Principle. There are a number of organizations that have set relevant standards. These include, but are not limited to nor intended to be a complete list:

• Media Measurement: Source of the content (print, broadcast, internet, consumer generated media) along with criteria used for collection; Analysis methodology – for example, whether human or automated, tone scale, reach to target, content analysis parameters
• Primary Research: Methodology – sampling frame and size, response rates margin of error, probability or non-probability, screening criteria; Questions – all should be released as asked (wording and order); Statistical methodology – how specific metrics are calculated; Identification of any potential biasing effects in the research itself or taking place in the broader societal context.