Archive for September, 2013

Guest Post: Vox Media, Hacking on Digital Ad Measurement with Chartbeat

September 18th, 2013 by Michael

Our guest blogger today is Michael Lovitt, VP of Engineering at Vox Media.

Last month, Vox engineers spent three days at Chartbeat’s NY office hacking on digital ad measurement. I want to share Vox’s take on the current state of online advertising and tell you how and why we’re working with Chartbeat to make things better.

The current state of online advertising

The state of digital advertising -- and brand advertising, in particular -- is suboptimal. In short, as an industry, we’re showing ads that readers don’t want to see and measuring using techniques that don’t tell us enough about whether ads are successful.

The good news, from our perspective at Vox Media, is that digital advertising has enormous room for improvement. We believe that by designing beautiful ads that delight instead of annoy, elegantly integrating these beautifully-designed ads alongside premium brands and content and for a premium audience, and employing measurement techniques that reveal a complete picture about who viewed an ad and to what degree readers engaged with and were influenced by it, that we can provide an advertising experience that does a better job of pleasing everyone involved: publishers, advertisers, and readers alike.

We’re making progress on all of the above, but solving these problems isn’t easy. It helps to have great partners.

Solving problems with Chartbeat

On measurement in particular, we were excited to work directly with Chartbeat because, like us, they’re convinced that the current state of digital ad measurement is not good enough and are actively working to make it better. Of course, Chartbeat is in the business of knowing how users engage with websites, and they’ve started to put that expertise to work tracking ads. (We had nodded our heads all the way through reading their recent blog post on the superiority of Engaged Time over impressions and clicks and awesome study showing a strong correlation between Engaged Time and brand recall.)

We’re also passionate (OK, addicted) users of Chartbeat’s products. Screenshots of important Vox milestones as visualized by Chartbeat litter our Campfire transcripts. These people do awesome work and it’s fun to hack with folks whose work you admire.

For all of these reasons, when Chartbeat invited us to attend one of their Hack Weeks, we immediately said yes.

Pre-hack planning

The Hack Week invitation was made at an opportune time, as Vox had just released an early alpha version of a new ad product metrics dashboard. The system was still in its early stages, but it was functioning in at least a basic way at every layer, and it served as a foundation for us to hack on.

We arrived at Chartbeat with a mock of a few changes to the dashboard -- we knew we wanted to start reporting, for every ad, both the Average and Total Engaged Time. As Chartbeat has shown, Engaged Time correlates well with brand recall, so being able to provide this level of insight to our advertisers and to ourselves would be one important step forward in moving beyond standard clicks and impressions

So we came with one concrete thing that we wanted to accomplish (having validated the idea with Chartbeat beforehand to make sure what we had in mind wasn’t crazy); beyond that, we were ready to play it by ear.

Hack, hack, hack

Here’s a quick rundown of what happened during our three days at Chartbeat:

Trei, Niv, Pablo, Aaron, and I arrived late Wednesday morning and kicked off with a short presentation to the Chartbeat team about Vox, and then settled into Chartbeat’s Stark Tower conference room and got to work.

We sat down with some of the Chartbeat team -- Harry, Matt, Wes, Shaun, and Alex -- and reviewed the dashboard mock. It turned out that Chartbeat’s existing ad product would be sufficient to record all the data we needed, but that a new API would need to be built to return that data back to us. Vox engineers got to work adding Chartbeat tracking to Vox ads, and Chartbeat engineers started building out the new API.

By the end of the first day, we had a Chartbeat-instrumented Vox ad running in production and a new Chartbeat API running on a laptop and returning data from Chartbeat’s production data store. From there, we could run our metrics dashboard in our dev environment and start to see real data flow in. Hurray!

On the second day, we solidified what we had built the day before. We wrote the front-end code to beautifully display metrics on the dashboard and got the Chartbeat API running on a server on the web so that we could push our metrics dashboard changes live.

Since we were making good progress on our primary goal, we also took some time to hack on Chorus, Vox’s publishing platform. We used an existing production Chartbeat API to integrate real-time Chartbeat metrics into the Chorus layout editor -- so that, for example, an editor on Polygon, when deciding what stories should be placed on the home page, can easily see the current visitor count on each currently-placed story.

The morning of the third and final day, we asked ourselves if we could get one more metric, Engaged Concurrents, integrated into the dashboard before the 1pm demo. It seemed feasible so we tried and got it done.

We showed off what we had all built to the Chartbeat team over pizza. After declaring total victory, the Vox crew said goodbye and stopped for celebratory drinks while we waited for our train back to DC.

Closing thoughts

We had a blast hacking with Chartbeat. It’s fun to make things with smart people, especially when they’re focused on the same problems as you are. Beyond ad measurement problem-solving, we enjoyed getting to know the team and talking shop: approaches to team organization, benefits and trade-offs of remote versus centralized workers, respective merits of modern JavaScript frameworks, and on and on.

Finally, we’ve been able to maintain post-hack momentum and have made great progress taking these projects to completion -- an important final step in any hackathon, and especially a cross-company hackathon like this one. Chartbeat has already released the APIs they built during our visit into production and we’re working this week at Vox to update the API client code in our metrics dashboard.

Now that we have this ad engagement data, what’s next? To start, we’ll be using this data internally to better understand how well our ads are performing -- by feeding it into our ad design process, measuring ad performance, and iterating. We’ll consider this data alongside other metrics that reveal how users are viewing and interacting with our ads. As we continue to take on the hard problem of optimally measuring digital ads, we look forward to having partners like Chartbeat at our side.

  PS- Check out Chartbeat CTO Wes Chow's guest post for the Vox Product blog.

Adweek Infographic: What Part of a Web Page Gets the Highest Viewership?

September 18th, 2013 by Juliana

Chartbeat is in Adweek this morning! After analyzing 25 million online user sessions, we made some surprising discoveries about where on the page your readers are spending the most time and which part of your page gets the highest viewability.

Click here for the whole Adweek infographic

 

Adweek – Breaking News in Advertising, Media and Technology

           

What I Learned from Redesigning Our Support Site

September 16th, 2013 by Alexis

Chartbeat | The Chartcorps-1 For as long as I've been part of the Chartcorps (the Chartbeat support, education, and strategy team), we've been hoping to redesign the Chartbeat support site. Luckily A few months ago, the Chartcorps teamed up with members of our Marketing crew, and decided to tackle a total overhaul of the support site. As a team of five we all shared the common goal of creating a well-designed, easily-navigated site that helped clients gather the right information quickly.

In beginning this task, we did what most do when they’re embarking on something new, we Googled. A lot. We researched support sites from all different brands. This helped us discover how our support site homepage could be designed, how easy (or not) it can be to find the right help content across different sites, and even how to intertwine separate products within one site.

We then took a look in the mirror – what was the current Chartbeat support experience like? From that we evaluated what to ditch, what to keep, and what to improve upon from the existing site.

We also reviewed  data on our existing support site, and saw that clients weren’t always surfacing the answers they needed. In addition to this, the site’s format wasn’t as intuitive as it could have been. So right away we focused on a key goal of making our help content easy to find as part of a larger, improved support or knowledge-seeking experience for our clients.

Breaking Down the Work

What’s tricky about creating a support site is that there are so many different moving pieces.  Here’s some of the initial tasks we needed to accomplish:

  1. Familiarize ourselves with our new site platform so we can make the most of it.

  2. Get on top of all things content-related: inventorying, revising, editing, and creating new answers and sections as needed.

  3. See what brands or companies are currently doing support well and pick their brains.

  4. Make key decisions about the new support site’s look and feel.

Check out our first (and extremely rough) wireframes of how we initially envisioned the the support site to look like and feel – you have to start somewhere!

wireframe

What Was Helpful

One of the best things about this experience was visiting SquareSpace. The SquareSpace Support team opened their doors (and arms) to us to give us insight into how they created their fantastic support site and how the team works on improving their customer support experience on a constant basis.

I know I mentioned data earlier, but I can’t emphasize enough how integral analytics were for this process. Beyond getting overall insights about the site, we used data to guide how we prioritized organizing the site’s help content. Having the numbers behind our content let us make insightful and informed decisions about how our support site content should be designed and made available to our clients.

What Was Hindering

As we continued to work on the site, we went through peaks and valleys. There were obstacles and significant delays, with two main problems occurring in the middle of our redesign:

  1. We couldn’t get our support platform to configure in a way that worked with our multiple products.

  2. With a wealth of knowledge-based content, figuring out where to begin was a daunting task.  At times it felt as though we were flying blind and it was hard to get insight into how other folks had approached building their own support sites.

However, after countless huddles, chocolate binges and last-minute pivots, we were able to create something we’re pretty excited about: support.chartbeat.com.

support site

What are the takeaways from this project?

  1. See what’s out there and learn from it. There are some great support sites already in existence, and exploring those help hubs certainly helped. Our meeting with SquareSpace really helped us because we learned from their experiences in running an effective site.

  2. Stay focused on our users’ perspectives and needs. How would clients search for that help tidbit? What sort of communication channels do they need on our site? How might a client misspell a feature name – and how can our site’s search function anticipate it?

Those are just a couple big learnings. And there will be many more lessons to learn as we hear from clients about what other content, help features, or design changes they need to access the knowledge and support necessary for using our products more effectively (and happily).

Check out our support site and let us know what you think via twitter or email– your feedback will help us make improves as we continue to iterate!

   

Chartteam Scenes (Summer 2013)

September 13th, 2013 by Cat

It’s been one hell of a New York summer and we’ve been busy enjoying the heat at Chartbeat. Here’s a quick summary of our adventures and all the fun we had when we weren't, you know, building things.

The team gets some serious training at Chartcamp. We’re Hunger Games ready now. The odds were definitely in our favor.

 

We kicked butt at softball (a few times) led by Coach “Gramps” Clarkson.

 

Not to brag, but some of us even conquered a half Ironman. The rest of us cheered them on the virtual sidelines.

 

We’re wrapping up the summer with happy hours and sailing, of course.

 BLOG | Scenes of Chartbeat (September) - Google Drive

 Want to get in on the action? Check out our Careers Page. We’re hiring. Apply to be part of the crew!

I’m Speaking at DataGotham 2013

September 11th, 2013 by Josh

datagotham_c0efd89d7bbe4db0ba7fc3608fe9b16d

Catch me speaking this Friday at DataGotham, a New York event dedicated to data science and those who work with data across all sorts of industries from startups to academia to local politics.

I’ll be talking at 2:40pm about a set of lab studies we’ve done to understand exactly how users read content, and then show how we use insights from those studies to power insights at scale across our network.

New York has a vibrant data science community that will be well-represented at DataGotham, and I’m excited to be presenting as part of it. If you’re not watching in-person at DataGotham, you can get tweet updates on my talk by following @Chartbeat.