An Update on Metrics and Reporting

Today we’re updating our metrics to give our partners and the industry more clarity and confidence about the insights we provide.

We know that having access to reliable metrics is important to the millions of partners who use our services to grow their businesses. As our products evolve to meet the needs of the people and businesses that use them, our metrics will also evolve.

Our goal going forward is to communicate more regularly about our metrics, so that our partners can focus on doing what they do best – serving their customers – with the best insights possible.

Increased Third-Party Verification
We believe strongly in third-party verification to prove the business value we’re driving for our partners, and we have a long history of working with global industry leaders like comScore, Moat, Nielsen, and Integral Ad Science (IAS). We’re now exploring additional third-party reviews to validate the reporting we offer partners. We’re also launching the ability to verify display impression data through our third-party viewability verification partners, including Moat, IAS, and comScore. This integration addresses requests we’ve received from partners for independent measurement of the amount of time ads are viewed on-screen.

For publishers, we’re partnering with Nielsen to include Facebook video and Facebook Live viewership in Nielsen’s Digital Content Ratings (DCR). This will give publishers access to third-party verification for video metrics and allow for comparable digital and TV metrics in Nielsen’s Total Audience Measurement.

Building More Measurement Solutions with our Clients
Our global Client Council has helped shape the direction of our products and how we approach measurement. Innovations such as conversion lift and mobile polling came out of direct conversations with our Client Council members. Given the Council’s valuable insights and the growing need for measurement standards tied to business outcomes, we’re now working with its members and other business and measurement executives to form a Measurement Council. We’ll announce more details in the coming weeks.

Regular and Clearer Communication on Metrics
We’ve created a new internal review process to ensure our metrics are clear and up to date as our product offerings continue to evolve. As part of this process, we’ll be communicating more regularly about updates we make. Communication will occur in a few ways, some of which are already happening:

1) In-product definitions. As we’ve routinely done in the past, we’ll continue to include updates in-product where partners buy ads or access our reporting. When we make meaningful updates going forward, we’ll continue surfacing them in-product.

2) Through our client teams. We’ll continue to communicate important updates to metrics that impact our partners directly through our partner teams.

3) Metrics FYI blog. We know how important it is to be open about meaningful updates we make to our metrics, so we’re creating a new channel for regular information on metrics enhancements. (This blog will be similar to our News Feed FYI blog)

Below is the first post in our Metrics FYI blog series, outlining the first set of updates we’re making. Please note that we do not bill clients on the potential under-reporting/over-reporting metric issues mentioned below.

Metrics FYI

Page Insights – Organic Reach
We’ve uncovered a bug in Page Insights. On one of our Pages dashboards, one summary number showing 7-day or 28-day organic page reach was miscalculated as a simple sum of daily reach instead of de-duplicating repeat visitors over those periods (see red circle in screenshot below). However, the vast majority of reach data in the Page Insights dashboard (marked green in screenshots below) was unaffected, including all the graphs, daily and historical reach, per-post reach, exported and API reach data, and all data on the Reach tab. The de-duplicated 7-day summary in the overview dashboard will be 33% lower on average and 28-day will be 55% lower; data in other fields is unaffected. This bug has been live since May; we will be fixing this in the next few weeks. It does not affect paid reach.

Additionally, we’re making an improvement to Page organic reach to match what we’ve done for paid reach. Reach counts will now be based on viewable impressions. On Pages, we’ve historically defined reach as a person refreshing their News Feed and the post being placed in their feed. For paid ads reports, we’ve moved to a stricter definition that only counts reach once the post enters the person’s screen (“viewable impressions”). With the stricter definition, we estimate that reported reach will be 20% lower on average. We’ll provide an update to all Pages in-product and via the Metrics FYI blog when we make this change, which will happen in the coming months.

Video – Measuring Completions
When partners upload their videos to Facebook, the full video length is recorded, but when the video delivers to people’s devices, the length of the video can sometimes be a fraction of a second shorter or longer. This occasionally happens when the audio and video track don’t line up, owing to differences between video players and devices. While someone may watch a video to completion on their device, the audio may continue to play for a bit longer. This particular issue caused us to undercount the metric ‘video watches at 100%’ (previously named ‘video views to 100%’). Moat found this and reported it to us. We are now updating how we read the video length to address this issue. This may result in roughly a 35% increase in the count of ‘video watches at 100%.’ For example, if ‘video watches at 100%’ were 1%, they would now be 1.35%.

Instant Articles – Time Spent
Instant Articles is a way for any publisher to create fast, interactive mobile content on Facebook. We also provide publishers with insights to help them understand how their content is consumed. We’ve determined that the average time spent per article had been over-reported by 7-8% on average since August of last year. This was caused by a calculation error: we were calculating the average across a histogram of time spent, instead of reflecting the total time spent reading an article divided by its total views. We have now fixed this issue.

Analytics for Apps – Referrals
In our Facebook Analytics for Apps dashboard, one metric called “Referrals” is miscalculated. This metric evaluates all posts produced by people via an app or website. We meant to count clicks that went directly to an app or website; however, we’ve also counted other clicks on those posts via the app or website, including clicks to view photos or video. We are working to fix this. Out of the referrals we currently report, on average about 30% are actually clicks to consume content on Facebook. For power users of this metric (top apps that look at this data in the dashboard most frequently), we found that referrals have been overstated by approximately 6% on average. Other measurements of referrals, such as those appearing in Facebook’s ads reporting tools, are unaffected.

Interest Lists – Follower Counts
Interest lists are a way to organize and view content on Facebook. Given low consumer usage of interest lists, we have decided to retire this feature, which will result in a drop in the total number of followers for profiles that created interest lists or were featured in lists. There are two reasons for this: 1) For people that created or were featured in an interest list, profile follower counts include the followers of the interest list; 2) People can follow someone from a profile and through an interest list, which means a profile’s follower number could double count that person. This update does not impact News Feed distribution or Page Insights. The impact to profile follower counts will vary, depending on the number of interest lists the profile created and was featured in. Most profiles will see a drop in followers of less than 5%.

Clarifying Metrics
We’ve been working for several months to improve reporting, and today we’re announcing several updates intended to make information clearer and easier to understand. Some of the updates we’re making are described below; for all we’ll be updating our in-product definitions.

• More descriptive names: We’ll make updates now and in the coming months to ensure our metrics are clearly named (e.g. “view content” → “website view of content,” and “video views” → “3-second video views”).
• Clarified calculations: We’ll clarify some calculations to make sure they reflect how our products are being used by advertisers. For example, with the launch of Canvas in February, we gave advertisers the ability to create an immersive native experience on Facebook. Within Canvas, they can also link to offsite content on Facebook’s in-app browser. While this experience is valuable to advertisers, the original intention of the Canvas View Duration metric was to capture time spent in Canvas. So we’re clarifying the metric calculation to reflect that it excludes time spent outside Canvas (linking to offsite content).
• More consistent definitions: We’ll also update our tips section and glossaries across all of our ads tools to make sure we are explaining our metrics in consistent ways (including noting which insights are estimates). For example, as we have rolled out new immersive experiences on Facebook like Canvas and lead forms, the link clicks metric has evolved over time to include clicks to those on-Facebook destinations, in addition to clicks to a website off of Facebook. We had updated the definition of links clicks in our ad creation tools, and we are now updating it across all of our ads tools. We’re also clarifying the description of our video views metrics (3, 10, and 30 seconds) to include the term “aggregate,” so as to more clearly explain it’s based on aggregate view time of at least the specified number of seconds.
• Better categorization: We’re making it easier for marketers to select certain metrics to include in their reports. To do this, we’re creating a way to customize the columns in reports to reflect how marketers set up their ads. This will let advertisers have a specific advertising objective map directly to the metrics we show, like checkouts, leads, or registrations.

These efforts are long-term investments, and we’ll continue to share updates on the Metrics FYI blog.