Chartbeat

At 5pm yesterday I was still recovering – yup, from the night before. That slightly dehydrated feeling, the craving for left over snacks in the fridge, and the sweet scent of stale keg beer are pretty solid signs of an incredible event. So let it be known, Chartbeat and Disqus know how to throw a party.

And throw one we did. With our two companies focused on a similar goal of getting publishers and brands to understand the importance of true engagement — with our recent Chartbeat Publishing for Ad Sales launch and Disqus’s emphasis on transforming time spent into time invested — we decided it was high time we paired up to tell this story together….over a few kegs and some Pies-N-Thighs deliciously deadly sliders.

So we called in a couple of friends to help us tell the story around engagement from their points of view:

1. Phillip Smith news innovator and publishing consultant lead the discussion and kept the rest of the gents in line.
2. Joe Alicata from the best real-time data service out there (please don’t tell me I have to clarify: Chartbeat) also bringing with him loads of experience leading product for ESPN.
3. Ro Gupta VP of Strategy for Disqus and fantastic partner in crime
4. Sam Slaughter VP of Content for Contently, an incredible company helping brands tell powerful stories
5. Stuart Schwartzapfel VP of Audience Insights for Big Fuel, who’s about the best advocate for the right kind of social and engagement measurement you could hope to find.
6. John Keefe Data News Editor for WNYC.org spreading the word about what his team has been built on – build amazing content people engage with, and the rest will come.

To those of you who were able to make it, thank you for coming and bringing your thoughts and questions. It was fun sharing drinks, laughs, bean salad, and geekery with you. If you’d like to be kept in the loop for our upcoming events, just shoot me an email and I’ll keep you posted. If you missed us this time, we’ll have all of it available for your viewing pleasure via a video of the talk in the coming weeks. We’ll kick it your way as soon as it’s available.

Until then, cheers to spreading the good word about engagement!

Last week, Chartbeat launched a new product, Chartbeat Publishing for Ad Sales. If you haven’t read Alex’s wonderful write-up of the inspiration and genesis of the product, you really ought take a few minutes to read it through. In a sentence, though, the product highlights exactly where on a site their readers spend their time reading, so that publishers can identify the ad positions that best capture their audience’s time and attention.

If you’re a skeptic like me, though, talk of Engaged Time around ads begs a question: readers come to a page to consume content, not ads — so, does higher engagement with a part of a page actually correlate with higher engagement with the ad that’s in view at that position?

When we started down the road toward building this product, my first focus was on answering this question, and I want to describe some preliminary work I’ve done to understand the interaction between Engaged Time and an ad’s impact.

Experimental methodology

At its most basic, the first goal of advertising is to capture a user’s attention — a person who doesn’t see an ad doesn’t even have a chance to interact with it. One simple measure of attention is brand recall — the ability of a person to recall the content of an advertisement some time after the ad is not longer in front of him — and it’s a well established metric for research on advertising’s impact. There has been a great deal of research on the effect of exposure time to an ad on brand recall for TV advertising [1], and other research has found a correlation between time on page and an ad’s effectiveness [2], so I decided to set up a simple experiment to measure whether the amount of time a reader is engaged on a page while a particular ad is in view affects their recall of an ad’s brand. The process went like this:

1. We created an a simple white page with a static news article and a single ad on the right rail, for a well-known national brand. No other images or headlines were on the page.

2. Participants were asked to read the article and told they’d be given a survey about the article’s content.

3. After a certain number of seconds, the screen was wiped and replaced with short survey: first we asked the participant to correctly identify the topic of the article, and we then asked them to identify the brand from the advertisement (the first mention we’d made of the ad at all).

We solicited 1,500 paid participants and randomly varied the amount of time participants were able to read before the screen was wiped — the only difference between each person’s experience on the page was the amount of time they were allowed to read for, which ranged from 5 to 20 seconds.

When the study started, I wasn’t sure what to expect, but as the numbers came in the results were clear: there was a strong correlation between Engaged Time and recall. Participants who read for 15 or more seconds were 25% more likely to recall the brand than those who read for 10 or fewer seconds. Even in this simplified environment, the amount of time the ad was in view had a dramatic effect on recall.

More complexity

Next, I wanted to test if the same would hold true on a more complex page. We repeated a similar experiment, this time with 1300 people. As before, each person was shown an identical page — but this time it was an actual live page from a publisher’s site. Participants were randomly shown the article for either 5 or 10 seconds, the screen was wiped, and they were asked to identify the content of the article and the brand from the ad.

As we’d expect for a more complicated page, recall was substantially worse across the board, but we saw the same trend as before — participants who read with the ad in view for 10 seconds were about 30% more likely to recall the ad’s brand than those who had the ad in view for 5 seconds.

Conclusions

In some sense, the results I’m describing feel obvious — we’d expect the amount of time a person sees something to have an effect on his memory of it. On the other hand, the impact of display ads is challenging to understand because they’re non-disruptive — a person can consume content while an ad is in view without necessarily consuming the ad. Traditionally we’ve had to rely on metrics like click-through rate to talk about impact, but the results we’ve seen so far suggest that time itself is a meaningful indicator of the performance of an ad.

There’s much work yet to be done to understand how time and advertising go together — how to design the creative based on how much time users will see it for, how much time different types of messages might need, how time interacts with more complex metrics like brand lift — and we’re actively working with academics and other researchers on further directions of study. For now, I’m excited to be one step closer to understanding the value of engagement on advertising.

Note: Check out Ad Age’s piece on our research.

Citations

[1] https://cdr.lib.unc.edu/indexablecontent?id=uuid:96863a34-efde-4726-9968-2cd7a511d036&ds=DATA_FILE
[2] http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=179598

Chartbeat CEO and sometimes polar explorer Tony Haile is featured in Fast Company today for four lessons he’s learned over four years at the helm of Chartbeat.

“What I Learned By leading a Startup – and Taking on Google” is a candid reflection of navigating common and not-so-common issues related to leading a rapidly-growing startup.

Tony discusses why it’s important to prioritize company culture, encourage individual autonomy, and look beyond a long resume. It’s solid advice for anyone with an entrepreneurial bone in their body. We’ve included one of his lessons below – get the entire piece here.

Let us know your reactions in the Comments section below or get in touch via Twitter.

From “What I Learned by Leading a Startup – and Taking on Google” by Tony Haile:

2. Experience matters. But so do intelligence, drive, and guts.
There’s nothing quite like hiring people with experience–those who’ve been there before, have made all the mistakes and know the path to follow. However, experience can have its limits and issues. Too often people will default to what worked at a previous company with a different culture, market, and set of challenges, leaving them frustrated when history doesn’t repeat itself.

The best kind of experience comes from those who are students of what they have done, who understand why the choices were made in their last company and the conditions required for success. It’s easy to mistake one set of experiences for the other, especially when interviewing, which is why I’ve learned to value curiosity, intelligence, and drive as much as a long resume. Some of my proudest hires at Chartbeat have been people with little direct experience in the roles they’ve taken on and yet have shown a speed in learning and improvement that still amazes me.

So yes, value experience but don’t let a hefty CV blind you to the opportunity to hire someone who is short on resume but long on everything else that makes a team member great.