I recently attended the fourth Measurement Summit and Conclave, a meeting of 50 thought leaders, academics, researchers and just plain “measurati”. Organized by the Institute for Public Relations Measurement Commission, the group convenes annually to help set industry standards and discuss the future of measurement in public relations.
I’ve previously written about how the Barcelona principles guide the standards for measurement metrics, such as mentions, impressions, engagement and awareness. The purpose is to ensure all practitioners speak with one voice.
The standards also help assure both the public and C-suite that we are accountable to our results and that bad math, bad coding or even a poor understanding of how these metrics are calculated won’t skew the numbers.
As the channels and tools in which we tell our stories constantly evolve, so do our methods for measuring them. The Commission serves to reevaluate our definitions and adapt them as the industry adapts.
The following are highlights from this year’s session as they relate to earned traditional and digital activities.
The Impression Enigma:
In the past, the Commission has frowned upon the use of impressions when tabulating the success of both traditional and digital programs, due to the wide variance in how impressions are calculated. But as I pointed out at the Summit, what’s critical is consistency – it’s imperative to speak the same language that our partners in advertising and marketing speak.
When calculating online story impressions, the best practice is to use Compete.com to determine the unique monthly visitors of a URL down to the most specific level available. For example, the difference between www.nytimes.com/business/dealsheet and http://www.nytimes.com is about 22 million impressions! Then, you must divide the monthly visitors number by 30 to obtain the daily readership, which can be folded into your impressions. The same rule of thumb works for blog posts.
Even for traditional media impressions, be sure to understand where your numbers are coming from, whether its Nielson or comScore. Keep in mind that even between these two recognized entities there can be major differentials.
The Commission had a spirited discussion on the concept of engagement and how it is evolving. The standard definition of engagement is any interaction with a brand beyond the exposure of an item, which can constitute likes, retweets, comments, shares, video views, etc. But what about indirect engagement that might occur when someone shares or talks about your content without you being involved?
An example is an earned traditional media story about your brand that is shared by a news organization on Facebook or Twitter and, as a result, is shared numerous times by that organization’s followers. Although a Huffington Post reader is engaging with your content through the media outlet, can that be considered an engagement with your brand?
The consensus was yes. As long as that was an objective of your campaign, shares of your content are considered a higher level of engagement. The group arrived at the same conclusion about the use of custom hashtags created by a brand but then used by social media users for their own original content.
New Trends in Natural Language Processing:
Coding sentiment for large numbers of traditional placements and social media mentions can be an arduous process but critical for measuring and benchmarking. Many have turned to partners who use automated tools to analyze sentiment, yet when the standards were first written the technology was not in place to accurately gauge sentiment.
However, there have been recent developments in Natural Language Processing that are greatly increasing accuracy. Several members of the Commission compared human coding to Natural Language Processing systems like Netbase and found, surprisingly, that the robots oftentimes outsmarted the humans in reading posters’ tone. The new confidence has led to a change in the standards, which is more accepting of automated analysis. Good news for us all!
PR measurement must be custom, but in line with accepted standards, which evolve over time. You also can read previous posts from this member of the measurati about other trends related to public relations measurement.