Real Measurement: Quantcast V. Alexa V. Compete.com

One of the core tenets of the internet, internet media and internet advertising is that everything is measurable. Advertising and targeting users has become data driven science and businesses are more efficient because of it. Wix (A Benchmark company) for example, tweaks the time, text and offers in its email marketing based on detailed conversion analytics gathered every minute, optimizing marketing at an incredible frequency.

What I find surprising therefore, is the disparity in data that we accept from internet measurement companies. Fundamentally, there are two ways to measure internet traffic and engagement: sampling and real data (which requires pixel placement). Sampling was the state of the art measurement for TV and Radio because that was truly all that was technically feasible. Nielsen built an enormous business out of sampling audiences for TV, retail shopping and anything else that needed to be measured. The question is: why should we settle for sampling in the internet era when everything can be measured. Openly on the web, Compete.com uses sampling data from ISPs and Alexa uses Alexa toolbar data which is necessarily skewed by who installs the toolbar.

This is how Compete describes its data collection methodology:

How We Get Our Data

Compete's data comes from a statistically representative cross-section of 2 million consumers across the United States who have given permission to have their internet clickstream behaviors and opt-in survey responses analyzed anonymously as a new source of marketing research.

For contrast, Quantcast uses a pixel to accumulate real full data. From Quantcast's methodology:

Direct Measurement

In a major advancement over traditional panel-based measurement methodologies, Quantcast couples machine learning with massive quantities of directly measured data to deliver detailed audience data in real-time for all forms of digital media including websites, video, widgets and advertising campaigns.

Using Seeking Alpha (A Benchmark company) you can see a clear example of discrepancies in Unique Users between real measurement (Quantcast for Seeking Alpha) and Sampling (Compete.com for Seeking Alpha)

Compete.Com estimates that Seeking Alpha has 1.8MM Uniques per month

Quantcast puts Seeking Alpha's US user # at 3.9MM Uniques Per Month

Quantcast, using a pixel on Seeking Alpha's site measures more than double the US users for Seeking Alpha than Compete measures using its sampling data and also suggests a significantly larger growth curve. But let's dig more. Most of the Alexa data is relative (that is relative to other sites) so it is hard to find apples-apples comparisons but Alexa's estimates on Seeking Alpha's US vs. International traffic vary pretty widely from Quantcast.

Alexa estimates that 67% of Seeking Alpha's users are in the U.S.

Quantcast knows that almost 83% of SeekingAlpha's users are in the US

I would love to compare Page Views (PV) and Page Views Per Visit (PPV) between Quantcast and Compete but I could not bring myself to pay the premium membership for the sampling data at Compete. Contrasting Alexa data and Quantcast data, you can see an almost 100% discrepancy in Page Views Per Visit (PPV) when looking at the Seeking Alpha Data:

Alexa estimates that Seeking Alpha's Daily PPV per user is under 3

Quantcast knows that Seeking Alpha's Daily PPV per user is around 8

As I started perusing these sites, I found a few nuggets interesting. First of all, you are not required to publish your Quantcast data publicly but it does provide a measure of transparency that is reassuring to readers, partners and advertisers. For example, I tried looking up Techcrunch on Quantcast but the Data was hidden by the owner. The Compete.com data for TechCrunch showed a slide in users but my assumption is that were we to have the real data, we would see a similar reversal to what Seeking Alpha showed. Huffington Post is available on both Compete and Quantcast, with Compete showing almost a 10% drop before the recent rise and Quantcast showing only up and to the right (I assume the big spike in traffic is Osama Bin Laden driven).

I think the conclusions are pretty clear as to the advantages of direct measurement yet many entrepreneurs who come pitch us quote the Alexa data in particular very frequently.