Do both, and you can do each better.

Measuring Our Impact

To find out more about PR, media measurement and social intelligence, I turned to Cision Information Specialist Ann Feeney. Feeney shared her insights on what’s possible, what’s practical and what’s in store in the measurement realm.
Grimes: What synergies are common to PR, social and media measurement — related to measurement standards, analysis techniques and business use cases?
Feeney: The lines between PR, social media, and other media measurement are blurring, which makes sense since ultimately, they’re all about whether messages were successfully communicated to the intended audience.
We’re also seeing more blurring between marketing and PR measurement, since while marketing success is measured in how many people purchased a product or service, it’s the total perceptions that drive the decision to purchase.
Corporate social responsibility (CSR) is a great example of how marketing and PR tie together and how you need both sides of the picture to understand the audience’s perceptions. For example, you’d want to measure how a company is perceived as a good steward of the environment or as responsive to human rights, and then to measure how much that affects the company’s financial success.
Grimes: Is PR, social and media measurement at a do-it-yourself point, or are the functions so specialized that most brands will continue to rely on agencies for assistance?
Ann FeeneyFeeney: That depends on the clients, their resources, their goals and their needs.
Many individual functions can be do-it-yourself on a small scale. For example, you can easily track who retweeted a particular message.
But understanding patterns over time, how different audiences across different media responded and comparing that to other messages that you’ve tweeted isn’t an easy do-it-yourself.
Tools for analysis and measurement are only going to get better and more sophisticated, and at least in the near future, more complicated to run and to support.
Grimes: Do you find that clients typically have a good understanding of what’s possible technically and how to find the best technical approach for their business needs?
Feeney: We have clients on every range of the spectrum. Some are leaders in the measurement world while others are just getting started.
Grimes: You've talked about the ways sentiment analysis is colored by emotional spectrum, intensity and action indicators. From your point of view as a researcher, how do sentiment ratings restricted to a positive/negative range fall short?
Feeney: There are several reasons.
  1. Even knowledgeable humans can disagree on the sentiment of a statement or group of statements. Depending on the scales they’re using, 85 percent agreement is about as good as it gets. A metric that has 15 percent variance at best isn’t a good stand-alone tool.
  2. There’s not always a reliable correlation between sentiment and business metrics. For example, you’d think that movie sales and social media sentiment about that movie would correlate very closely, but across several studies, the correlation isn’t consistent.
  3. Studies also show that different kinds of emotions have different kinds of effects. Anger, for example, is more contagious than many other emotions, according to various research studies.
So if you’ve got an issue that’s making people angry, that’s going to spread more than something that makes them sad, both in terms of content and the spread of the emotion, even though they’re both negative sentiments.
But most importantly, by itself, positive and negative sentiment doesn’t inform action. Practical research has to answer the question, “So what?” and sentiment analysis as a standalone doesn’t answer that question.
Grimes: What do you mean by “action indicators”?
Feeney: Action indicators show what people are saying that they’ll do in response to something or what they want an organization or person to do.
Grimes: How do you identify likely actions in text sources?
Feeney: If you’re examining a specific topic, such as the Charleston (S.C.) church shootings, you use your knowledge of the event to look for specific concepts. For example, mentions of the Confederate flag or gun control are very likely to be associated with action indicators.
You can also look for verbs such as should, ought to, or must, if you’re looking for overall analysis or to find less obvious indicators. You can also look for future-focused verbs such as might, would, could or will, to measure possible future actions.
Grimes: Can you provide a quick example or two of success stories — organizations that realized positive ROI through text and social analytics?
Feeney: The 2012 Obama campaign used some of our tools to track issues in the swing states and understand which messages resonated most with voters in those states. It combined demographics with social media to identify persuadable potential voters.
A pharmaceutical company analyzed what people who want to quit smoking are saying online. These smokers felt that nobody was acknowledging that quitting is hard and that they needed support and understanding. The company focused its messaging on those aspects and got a definite increase in sales.
Grimes: What advice do you have for organizations that want to ramp-up or expand their measurement efforts?
Feeney: The single piece of advice I’d give is actually from the nonprofit evaluation field. Develop your logic model of the change you want to create.
If you aren’t measuring the right thing, then you can do the most sophisticated analyses from the cleanest and most complete data in the world, and it still won’t be the best answer for you.