Epic fail: Video view fraud detection

Download PDF file: How to save advertising dollars on Facebook and YouTube.
2015-09-28 Update thanks to Rubén Cuevas,

Fake views of ads by "bots" cost advertisers more than $6.3 billion US globally during 2015.

Data show, video fraud-detection on DailyMotion, vimeo, YouTube and others fails to filter out invalid traffic properly.
 
Here I distill our knowledge into 3 takeaways.

Check out what Sir Martin Sorrell WPP has to say about the matter.
According to Media Rating Council (MRC) and IAB (Interactive Advertising Bureau) standards, a viewable impression of a digital ad occurs when 50 percent of an ad’s pixels are on screen for one second.

In December 2014 Google published data regarding display ads in browsers (desktop and mobile). The study revealed that 56 percent of the display ads it served on its own and others’ sites never appeared within view on someone’s screen.

Nobody really knows for sure how Google or any other video platform or ad server come up with these numbers. For instance, Google provides explanations of what one should look for in these numbers it serves advertisers about their ads. How it collects them is, however, not explained.

1. What is the challenge?

The US Association of National Advertisers (ANA) released a report in December 2014, which estimated that

  • 23 percent of video ads, and
  • 11 percent of display ads

are viewed by “bots”. These are computer programs that mimic the behaviour of an Internet user.

The ANA estimated that this would cost advertisers about $6.3 billion US globally in 2015. This is a concern for two reasons.

1. Adertisers are spending ever larger amounts of money across both display and video advertising (see graphic below), and

2. Spending for video ads is estimated to grow 21.9 percent compound annually from 2015 to 2020 (US data) (see also online video celebrities – chart below).

2. Google and Facebook want a larger slice

Google and its YouTube platform want to garner the largest share possible of this growth in video advertising. Nonetheless, the competition will surely want to prevent this.

In April 2015, Facebook boasted it had over 4 billion video views each day. This number continues to grow.

For now, YouTube data suggest many more videos are viewed daily on its video platform than on Facebook.

For Google, display and video ads create tons of cash for the company, but things are changing. For instance, the rate for pay-per-click ads has been dropping (view chart as shown below). Google explains this was lower rates on YouTube than its other platforms.

Join the 3,000+ organizations using the DrKPI Blog Benchmark to double reader comments in a few months while increasing social shares by 50 percent - register now!

Some suggest that in the US, millenials spend nearly 60 percent of their time watching movies on either a smartphone, tablet or desktop/laptop.

To keep advertisers pouring more money into video ads, however, Google and Facebook have to up their game. Accordingly, both must provide strong evidence that their fraud detection systems work. Until fraud detection works, three things must be addressed as outlined below.

3. Focus on not getting charged for invalid video views

Each video platform wants to charge advertisers for video ads according to whatever the market will bear. In turn, advertisers want to keep costs for ads down, but this is becoming a challenge.

Apparently, some companies offer tens of thousands of YouTube views for as little as $5 US. Such data could in part help explain why 23 percent of video ads are viewed by fake consumers.

Of course, no advertiser wants to pay for these “views”.

How does one avoid paying for fraudulent views?

That is difficult to say, because…

Filtering invalid traffic before advertisers are ever charged is not getting easier.

Recent research sheds light on this important issue. Researchers uploaded two videos to each of five video platforms (YouTube, DailyMotion, Myvideo.de, TV UOL and Vimeo).

They bought ads on these platforms, which targeted the videos they had previously uploaded. Then, they directed their “bots” to these videos.

What are bots?

EXPLANATION: What are bots?

Bots are used by DrKPI, Google and Qwant Search to crawl the web.

They are little programs that allow DrKPI  to collect data about blog entries (e.g., text, data of blog entry, etc.).

Google uses bots to index webpages. Bots can also be computer programs that mimic the behaviour of internet users viewing, e.g., a video ad.

About 60% of internet traffic is due to bots.

Each platform’s two videos were visited by the bots about 150 times. The researchers explain in their paper that the bots used were far from sophisticated tools as cyber criminals might use. Nevertheless, the results are worrisome for advertisers.

If detection mechanisms work properly, marketers do not have to pay for ads on YouTube viewed by robots.

Data show that YouTube seems to have the best fraud-detection mechanism of the five platforms tested. It was followed by DailyMotion.

YouTube’s fraud detection tool identified 25 of the 150 bot visits to a video as real users viewing the video.

This means in 16.67 percent of cases, YouTube wrongfully identified a bot or robot to be a human watching a video.

♥ Curious? Join 1,500+ other subscribers to this blog’s newsletter and read on!

What is most unsettling, however, is that Google charged the researchers for 90 of the registered fake views. This is a 60.67 percent error rate!

So what is the bigger problem:

1. That YouTube wrongfully identified 25 robot “views” to be humans out of 150 times the video ads were “seen” = 16.67 percent error rate, or

2. Google AdWords, instead of charging for the 16.67 percent of views wrongly identified as humans by YouTube, deciding to charge for 90 views done by robots =  a 60.67 percent error or false positive rate?

How could YouTube’s false positive-rate be so inflated? The process of counting views (i.e. public view counter and number of counted and monetized views) is opaque on YouTube.

Thanks to Rubén Cuevas for pointing out: “YouTube has two different mechanisms in place to discount views for the:

public view counter, and also the
monetised view counter”

Important is here to understand as Rubén pointed out to me, the public view counter seems to be more strict in the detection of fake views.

This is to say YouTube increases the count, and therefore, charges the advertiser for even more fake views than the public view counter would suggest.

Read the research findings in detail:

Marciel, Miriam; Cuevas, Ruben; Banchs, Albert; Gonzales, Roberto; Traverso, Stefano; Ahmed, Mohamed and Azcorra, Arturo (July 2015). Understanding the detection of fake view fraud in Video Content Portals. Retrieved September 23, 2015 from http://arxiv.org/abs/1507.08874

Check out the FT article for non geeks, including comments by the researchers left here:

Cookson, Robert (September 23, 2015). Google charges marketers for ads on YouTube even when viewed by robots. Financial Times, p. 1. Retrieved, September 23, 2015 from http://www.ft.com/intl/cms/s/0/53ac3fd0-604e-11e5-a28b-50226830d644.html

3 takeaways: Focus on verifiability of video views to fight off deception.

1. Better process transparency for fraud detection
Having over 15 percent of bot views identified as “real” is a high error rate. While this is bad, YouTube is better than the rest.

YouTube seems to use a sufficiently discriminative fake view detection mechanism, but this applies only to the public view counter.

For the monetized view counter (i.e. those for which advertisers get billed), YouTube seems to ignore this mechanism for discounting fake views (see section 3 above – verifiability).

This is, of course, unacceptable for advertisers. Moreover, it makes the process of how YouTube detects these deceptors totally intransparent for advertisers.

Bottom line: With the help of third party verification, this challenge should be resolved quickly.

2. Improve measurement and use a set of standardized metrics
Even with third parties verifying numbers for advertisers, if our KPIs (key performance indicators) are not comparable we are stuck. For instance, Facebook defines a “view” as someone watching a video for three seconds or more. Others like YouTube talk about around 30 seconds before counting.

These different standards make it difficult for advertisers to get a clear feel and comparable numbers across platforms. Thus, even focusing on numbers, as Google suggests, is of limited value.

Bottom line: Define and agree upon the metrics used by the advertising industry. Make them comparable across social networks and video platforms.

3. Establish third party collecting, verifying and auditing of numbers
Facebook has followed the practice of self-reporting on viewability of ads, pages, reader engagement, and so forth. But as Volskwagen’s #dieselgate shows, self-reporting is always vulnerable to misuse, sloppiness and abuse of the system.

Bottom line: We need third party collecting and verification of numbers. Such efforts must in part focus on minimising charges for advertisers when ads are viewed by robots.

Eliminating fraud in online advertising is key

You are supposed to count the actual number of measured views of a video ad. Ergo, filter out invalid traffic from bots.

In December 2014, the ANA/White Ops study identified 23 percent of video ad impressions as bot fraud. Combine that number with the results from data reported here, and this means:

Google AdWords takes at least 60.67 percent of the 23 percent bot fraud views on YouTube and charges advertisers for them.

Thus, it follows that advertisers pay for at least 14 percent of video ads not viewed by humans!

The lack of transparency, standardized metrics and a regular audit of how video platforms handle fake ad views costs advertisers dearly.

Accurate metrics matter. For the first time ever worldwide mobile advertising will overtake print in 2016 ($71 billion US versus print shrinking to $68 billon US).

As well, social media advertising will top $25 billion US this year. Facebook is expected to take the biggest slice, more than $16 billion US. Instagram will account for “just” $600 million US.

Advertisers are justifiably wary and suspicious. Based on the above predictions, we better make sure that we pay only for those imprints, views, etc., that were executed by humans and not robots. Will #GoogleAW2015 tell us more about how YouTube plans to address this issue? Not really.

Download the checklist as a PDF (320KB file).

Interesting read

a) More content about advertising and viral content
b) Google: hidden ad costs
c) IAB’s efforts to establish a more trustworthy supply chain
d) YouTube frozen views
e) YouTube search for counted views – zero information provided
f) Facebook partners with Moat to verify video ad metrics

Benchmark and test your blog – for free – right now

What is your opinion?

Now that you have read “Epic fail: Video view fraud detection“, I would like to ask you a question or two.

– As an advertiser, how do you deal with this issue? Please share!
– What type of video advertising works best for your business?
– What do you know about Facebook’s handling of this challenge?

More about advertising fraud

Receive email updates on the latest
Strategy / Marketing / Trend research.
SIGN UP ››

Urs E. Gattiker

Professor Urs E. Gattiker - DrKPI is corporate Europe's leading social media metrics expert (see his books). He continues to work with start-ups. Urs is CEO of CyTRAP Labs GmbH.

17 thoughts on “Epic fail: Video view fraud detection

  • 28. September 2015 at 8:57
    Permalink

    I just found another paper that is interesting in this regard.
    Published by ACM but available also for free….

    It addresses the algorithm issue in more detail than the above paper.
    It also outlines how these proposed algorithms can be used for reducing fraud regarding views and how advertisers are being charged.

    Chen, Lian; Zhou, Yipeng, Chiu, Dah Ming (December 2014). Fake view analytics in online video services.
    ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) – Special Issue on MMSYS 2014
    Volume 11 Issue 2, February 2015
    Article No. 44. Retrieved, September 22, 2015 from http://arxiv.org/abs/1312.5050

    Reply
    • 28. September 2015 at 9:07
      Permalink

      Dear Peter

      Thanks so much for this link. I had not known about this one yet.

      I added this screenshot from the paper. It has nice explanations on how things work. See image below.

      On p. 18 of their paper the authors write:

      “Based on the “ground truth” we gathered from the data with user IDs,we check if their IP addresses are the fake view IPs detected by the TSVM classifier. For performance evaluation, we observe that 99.08% of fake view IPs with users IDs are discovered in the fake view IP detection process. The TSVM classification performs well.”

      So why have these method not been used to reduce fake views?
      Videos with lower entropy than normal videos are suspected to be fake view videos

      Reply
    • 28. September 2015 at 14:26
      Permalink

      Dear Peter,
      I am Rubén Cuevas, co-author of the paper on youtube and other major portal.
      The work that you are indicating is properly referenced in our paper.
      As you point out is a very interesting work, compelementary to ours. They use a trace from a video portal in China and discuss different ways of identifying potential fraudulent activity. Our study tries to auditing the fraud detection systems of major portals.

      I would recommend anyone interested in the topic to read both papers.

      Thanks

      Reply
      • 29. September 2015 at 8:47
        Permalink

        Dear Rubén

        Thanks for pointing this out to us. What I find interesting is that your paper and theirs are, as you point out, complimentary. They address:

        1. Chen, Lian; Zhou, Yipeng, Chiu, Dah Ming (December 2014) use a trace from a video portal in China and discuss different ways of identifying potential fraudulent activity.
        2. Rubén and colleague’s study tries auditing the fraud detection systems of major video portals.

        Chen and colleagues try to identify the fraudsters where they come from and so forth, thereby illustrating what portals should do to become less vulnerable against fraudulent ad views.
        You in turn focus on checking how well the video portals are handling this threat right now.

        BOTTOM LINE
        One can conclude from the above that there are tools to improve detection of fraudulent activity.
        However, Google and other video portals manage to charge you for these bot views as well.

        Great way for doing business if you can get away with it. 16 ad views out of a 100 are invalid / incorrect / faulty ones since they represent views by robots instead of humans.

        So should we conclude that the owner who uses such a charging model is unethical? Others might say video portals exploit an advantage unfairly that costs advertisers dearly. Need I say more: Not acceptable.

        Thanks for commenting on this Rubén

        Reply
  • 28. September 2015 at 9:31
    Permalink

    I forgot.
    You can get the paper also from the ACM Digital Library: http://dl.acm.org/citation.cfm?doid=2739966.2700290

    It is the same as the one posted on arxiv though.
    Interesting is the following quote from the conclusion of the paper:

    Our study is focused on fake views caused by robots generating fake video requests or reports. Our study found effective methods based on the use of IP entropy and video entropy functions, together with other strong features such as publish date of a video. We found an interesting way to use IP entropy and video entropy together to make our technique more effective. We also report some fake view statistics found in a real-world online VoD system.

    Another interesting paper is the one cited by the authors, namely:
    A. Lakhina, M. Crovella, and C. Diot. Characterization of network-wide anomalies in traffic flows. In Proceedings of the 4th ACM SIGCOMM conference on Internet Measurement, pages 201–206. ACM, 2004.

    Reply
  • 28. September 2015 at 14:31
    Permalink

    Dear Urs,
    Thanks for your post. I think you made a very good summary of the of the current landscape about advertising fraud for not-technical experts to properly understand it.

    I would like to suggest one correction however. This would help further improve the accuracy of the message in the post with respect to the following statement:
    “YouTube applies mechanisms to discount fake views from the public view counter. However, not from the monetized one.”

    I would rather say that YouTube has two different mechanisms in place since in both cases (public view counter and monetised view counter) views are discounted.
    However, based on our results, the public view counter seems to be more strict in the detection of fake views.

    Reply
    • 28. September 2015 at 16:05
      Permalink

      Dear Rubén

      Thanks so much for commenting here about the blog entry. I wanted to make 2 comments:

      1. as you have most certainly seen in my previous reply to a commentator I linked to another paper that also addresses this issue in a technical way – algorithms to detect fake views
      Of course you tested how well the platforms’ algorithms worked, these authors developed algorithms that work well.

      2. I inserted your change as per your suggestion in the blog entry. YES this is an important mistake on my part. Thanks for pointing out this error.

      Cordially
      Urs

      Reply
  • 28. September 2015 at 14:37
    Permalink

    Dear Urs,

    This is Rubén Cuevas, co-author of the work.

    Regarding the next steps I fully agree with you in the need to define independent third party auditing on the antifraud systems implemented by video portals, but also in other advertising contexts (social media, display ads, search ads…).

    The goal of our team in the mid term is building such a system. As we have shown in our work we are in a good position to do so, since we have already defined a robust methodology for this purpose.

    However, it would be very interesting for us to understand the interest of the advertising industry on supporting us on this endeavour.

    Reply
    • 28. September 2015 at 16:21
      Permalink

      Dear Rubén

      Thanks for the feedback. So we agree on the issue with needing independent third party auditing .

      Of course, I also agree with you that such work has to go beyond video ad views. It should also address display ads, search advertising as well as inserting stuff on social networks like Twitter, Facebook or Instagram.

      I would guess that any large advertisers would be interested, such as Unilever.

      In a recent FT article, Sir Martin Sorrell, chief executive of WPP, was mentioned as having pointed out that Google had to get its act together regarding fake views of ads. He even believes that unless Google and other advertising platforms deliver, “… marketers will shift their focus back towards traditional media.” See: http://www.ft.com/intl/cms/s/0/f9da727c-6207-11e5-9846-de406ccb37f2.html

      What you say to that …. if they put their money were their mouth is… your project should get the necessary funding.

      What you think.
      Greetings
      Urs

      Reply
  • 28. September 2015 at 16:31
    Permalink

    Dear Urs,
    Thanks for your comment. Hopefully we will get contacted by these big companies soon and get to it. We are very much looking forward to it.
    Fraud is a fundamental problem of the advertising industry and we would like to be part of the solution to it.

    Reply
    • 29. September 2015 at 7:25
      Permalink

      Dear Rubén

      You might have to contact Sir Martin Sorrell, chief executive of WPP, or maybe Dimitri Maex is Managing Director of OgilvyOne New York, Ogilvy & Mather’s Direct and Digital operations (he co-authored Sexy Numbers).

      They both have been known to voice their opinions about this situation and how independent third-party analytics could help. Just refer to Martin’s comments in the FT (see my comment above with link) and tell him who you are. I am sure he will be curious.
      Good luck.
      Urs

      Reply
      • 29. September 2015 at 12:50
        Permalink

        Rubén
        I think this is interesting what Facebook released yesterday:
        https://www.facebook.com/business/news/ad-week-2015-announcements

        “Marketers can plan a campaign across TV and Facebook with a total TRP target in mind, and they can buy a share of those TRPs directly with Facebook,” …

        “Then, Nielsen’s Digital Ad Ratings measurement system can verify Facebook’s in-target TRP delivery, and Nielsen’s Total Ad Ratings system can verify the TRP delivery for Facebook and television combined.”

        ===> metric called target-rating points (TRPs) read more here: http://www.marketing-metrics-made-simple.com/target-rating-points.html

        Again, it is verified by a third party. Unfortunately, only Nielsen can offer the service. This limits choice for advertisers greatly.

        Of course, how Nielsen and Facebook account for fake display counts – bots in action – is not explained. Again leaving advertisers open to this abuse.

        Reply
        • 5. October 2015 at 19:58
          Permalink

          Apple has made it easier to build ad blockers for the iPhone and iPad.
          Advertisers are not happy.
          But ad blocking can can burn through users’ data plans more quickly than they might expect. Else it slows down the web experience.

          So which sites are the worst? The New York Times did a test of the top 50 news site home pages on the iPhone.
          Incidentally, the NYT does quite well in the test… Guardian beats them all.
          Here are the news sites that use up the most data when you load them on your iPhone

          Reply
          • 6. October 2015 at 19:01
            Permalink

            Dear Urs

            You forgot the website regarding the above data which I post here

            http://www.nytimes.com/interactive/2015/10/01/business/cost-of-mobile-ads.html

            Of course, news websites are supported by online ads, and if enough people block the ads the sites may struggle. Ad blockers can also have technical downsides, sometimes causing websites to load erratically. In one of our tests, one website crashed repeatedly when an ad blocker was turned on.

            Looks like sites may fight back by possibly not letting us see their content…. or in the above quote … letting the site crash on my mobile.

            Thanks

          • 6. October 2015 at 19:19
            Permalink

            Peter

            Thanks so much for pointing this out to me. Incidentally, the ad-blocking threat is growing and the advertising industry is very unhappy with the Nielsen KPIs as well. Probably one reason why WPP, the advertising group that holds a minotirty stakes in ComScore and Rentrak, is pushing for these two to go together. WPP has been pushing for more accurate audience ratings, partly because two of the largest platforms for online viewing – Google and Facebook – have their own rating system for videos.

            And no, you cannot have players refereeing the match … i.e. auditing their own numbers. That is, of course not satisfactory as my blog post tries to demonstrate.

            Read more on the Financial Times: http://www.ft.com/intl/cms/s/0/df072c9c-686f-11e5-97d0-1456a776a4f5.html

            And

  • 10. October 2015 at 18:19
    Permalink

    Urs

    Thanks for this post and mentioning me and WPP. Appreciate this. Fake views are a problem for advertisers.
    These concerns must be addressed.
    I concur with you, a third party audit of the numbers is one viable strategy.

    THANKS
    from my … device

    Reply
    • 10. October 2015 at 18:33
      Permalink

      Dear Martin
      Thanks for your feedback on this blog post. Appreciate you taking the time.
      I think Ruben’s approach is a valuable one to test if these methods are working properly.
      I look forward to see how this will further develop.
      Thanks for sharing
      Urs

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *