The likes of Facebook and Google are no stranger to getting a public kicking when it comes to measurement blunders, but are some of the traditional methods of measuring ads really that great and is there a case of throwing stones from glass houses?
In mid–2016, it emerged that Facebook had been over–inflating video view metrics for around two years, fuelling the debate around digital players’ walled gardens and the need for independent third–party measurement. It wasn't the last and soon after, Facebook was back in the news for another metrics crisis.
The issues this time included the wrong allocation of reactions on streaming video, a discrepancy between Like and Share counts on mobile search, and a change in the way it calculated potential and estimated reach and potential audience size. Facebook took action and pledged to be more open as a result.
The social media platform wasn’t alone in the doghouse after the Media Ratings Council (MRC) in the US suspended Google DoubleClick for Publishers over the way it measured mobile web served impressions for publishers.
Last year also saw Facebook tell advertisers they could reach millions more millennials than existed in almost every leading economy, including seven countries in the G8. Needless to say, confidence in the robustness of Facebook and Google metrics has been shaken.
The news went down well with those sitting on the more traditional side of the media fence keen to leap on the digital duopoly for its errors, and those feeling taken advantage of by the money–siphoning tech giants. There is plenty of stone throwing from the traditional players when the digitals mess up. At the time, Seven's Tim Worner remarked that media buyers and marketers “would kill us” if the television industry misreported in the same way as Facebook had been found to have done.
AdNews March edition:
This first appeared in print:
The industry is edging towards cross–platform measurement and measuring any channel is becoming a quagmire of complexity. But, are the established systems for TV, radio, outdoor and beyond any better, or are their flaws just more familiar?
The rapid conversion of static to digital billboards has been a major advancement in the out of home (OOH) sector. However, issues have arisen about viewability and many have conceded the outdoor audience measurement system, MOVE, hasn't kept up with the transition to digital, leading some to question the accuracy of campaign reporting.
Others have argued that the radio ratings system is far from perfect given it’s still based on householders recording their listening habits in paper diaries. While the number completing e–diaries has increased, some still scoff at the dated notion of pen to paper recording.
But, it's worth noting late last year the Commercial Radio Authority (CRA) did announce a new multi–million dollar Measurement Innovation Program (MIP) to explore the potential for a hybrid future for radio measurement.
TV shows live or die based on viewership data and while OzTAM’s TV ratings are seeing estimates drawn from viewing behaviour in a panel of 10,000 people each day, some argue the panel should be larger. And they say the utility should include options to allow people to register when they are streaming. Netflix and Stan, for instance, are not captured in industry ratings.
It’s no secret that digital metrics are under scrutiny, but how do the more established ratings and measurement systems stack up? Do they still offer a high standard and represent the industry or are they out of date? We put it to four top industry execs as part of AdNews in-print March Big Question feature:
Monique Perry • managing director of media Nielsen:
Digital metrics, like measurement in general, should always be under scrutiny. The reality is that we are in a world where devices change, browsers are upgraded, search algorithms shift, and new content platforms and technology are designed and released daily.
As a result Nielsen, in conjunction with the industry, is constantly challenging and devising new ways to best measure consumers’ behaviour in a digital landscape. Today, the core metrics which can best tell the story of the digital consumer are unique audience (people) and time spent (duration).
Digital standards in measurement will continually evolve with input from global bodies like the MRC and local partners like the IAB. Our commitment at Nielsen is to be transparent along this journey of innovation so the market knows what we are measuring and what we are working to solve. Importantly, at the heart of any audience measurement solution there needs to be independence, and it needs to measure people.
The digital consumer is both the most measured and hardest to measure. Nielsen is on a journey of innovation to combine its rich history of media quality panels with the new world of census tag inputs and very large third–party samples.
Media quality panels, when directly measuring devices and representing the population, are the most important source of truth and calibration. Volumetric data captured from tags or SDKs across all content types is vital for obtaining a complete view of all activity.
Then we can layer in samples from large third–parties to provide the scale necessary to capture increasingly fragmented audiences or smaller time increments. These three inputs combine in digital measurement to provide daily audiences, across all devices, both on– and off–platform, and for small or large sites.
Mark Frain • chief sales and marketing officer MCN:
Put quite simply, TV will not lose its mantle when it comes to measurement and reach. TV still has what we refer to as a gold standard measurement system, given credibility by the fact that it is independently audited and globally recognised.
Apart from being able to reach specific demographics, TV provides full alignment as a currency for the entire industry to operate within, from media owner through to agencies, and ultimately, advertisers.
As the proliferation of premium, broadcast content is distributed on many different platforms, there is no doubt TV’s measurement will expand and evolve, whether that be a further evolution of the VPM ratings or an alternative method. Not to mention the impact of new cutting–edge data management solutions being developed to aide with multi–platform measurement.
It will mean TV measurement overall will improve and grow more granular, offering genuine attribution models based on real consumer spend data. This essentially will emulate what MCN has done with our Multiview TV audience measurement panel, which is based on the data of 200,000–plus anonymous homes.
As the multiscreen TV industry today, we sit in a privileged position with a mixture of panel based, set top box and login data that will ultimately get to a place where we can tell a true multiscreen story to the industry across both age, gender and rich consumer segmentation.
This first appeared in print:
Gai Le Roy • director of research IAB Australia:
The established currencies of traditional media are still relevant in attempting to estimate their audiences. However, components of their systems are definitely starting to struggle with the reality of both current media behaviour and an increasingly sophisticated and demanding buy side.
Audience sizes are changing, content is being consumed across multiple platforms, ad opportunities will increasingly differ for different audience members and basic demographic information will not suffice on its own. 2018 will demand that all media (including digital) develop their metrics and systems to be able to track increasingly targeted campaigns through new buying methods and assist in providing meaningful metrics that truly help marketers assess the impact, not just the reach of their campaigns.
However, many traditional media companies are challenged by high costs with shrinking or flat ad revenues. As audiences increasingly consume media in non–linear and digital format, it will be very difficult and expensive to change legacy systems — particularly with stakeholders who have a vested interest.
Ratings systems that rely on self–reported behaviour will struggle to cope with increasingly fragmented behaviour and provide the depth of information that advertisers demand. Instead, passive measurement through panels or other industry approved data sets is needed to provide scale and granularity.
The IAB looks forward to seeing more collaboration between different ratings systems and industry bodies to provide the market with meaningful data that reflects our complicated cross–media world and provides agencies and marketers with a more cohesive view of their media opportunities. We’d like 2018 to be a year of collaboration, common language and cross–media measurement.
But, this must reflect the whole media community and not just those driving the ship.
Within IAB’s deed for an independent third–party Australian digital audience measurement system (currently operated by Nielsen), it is mandated that the whole market is measured whether they are an active participant in the currency or not. Obviously those players who chose to be measured have more granular and accurate data, but this ensures we reflect the market reality.
I’d like to see this approach adopted across all mediums and media.
Joan Warner • CEO Commercial Radio Australia:
Radio audience measurement does stack up and is continuing to innovate. Radio is in the enviable position that it is ubiquitous; it takes place across time, place, and now platform. But as a result, it makes capturing all radio listening a challenge.
The industry is continuing to innovate the measurement of listening to ensure we capture as much listening as possible. In the last 12 months, Radio Audience Measurement (RAM) provider, international research company, GfK, conducted exploratory testing and a trial of radio audience measurement via a smartphone app. Prior to the app tests, a wearable electronic meter watch was also tested by GfK.
The detailed and extensive testing found the app and the watch do not capture all radio listening either, including — significantly — when the participant was using headphones. Plus, there were significant respondent non–compliance issues with metering technology. The naysayers don’t want to hear it, but the tests did demonstrate that the pattern of listening captured by both the watch and the app is consistent with the diary methodology.
Listening trends across the major parts of the day did not differ and this consistency was also found across gender, age, and content genre. As well as a robust validation of the diary survey data, the tests have provided valuable insights to inform the investigation of a hybrid model which may cover, for example, the use of electronic data to supplement and enrich diary data.
Last year the industry, together with GfK, announced a new multi–million dollar Measurement Innovation Program (MIP) to explore the potential for a hybrid future for radio measurement. Our objective is for Australia to add to the most robust radio ratings system of 60,000 participants each year and develop a world–first radio audience measurement system that uses a suite of tools for the Australian radio market, informed by facts to produce an even more holistic view of Australian radio listening.
The March issue of AdNews has hit desks. You can subscribe to the print edition or download a digital version here. Not all content appears online, some is exclusive to the magazine, so the only way to read it all is to get a copy - don't miss it.
Have something to say on this? Share your views in the comments section below. Or if you have a news story or tip-off, drop us a line at adnews@yaffa.com.au
Sign up to the AdNews newsletter, like us on Facebook or follow us on Twitter for breaking stories and campaigns throughout the day.