How to Leverage Real-Time Data to Develop an A/B Testing Strategy

How do you measure your visitors? Maybe you’ve lived by pageviews, monthly unique visitors, or ‘click’ measurements on various elements on a web page. Or, maybe you use a more sophisticated metric, such as Chartbeat’s Engaged Time.

At the end of the day, we’re looking to get a better understanding of our audiences. We can then leverage that knowledge to provide great content consistently, along with better overall experiences. However, what’s the next step for a publisher, armed with powerful real-time analytics? How can that governing metric, such as Engaged Time, be improved in a systematic way? Enter website optimization. Like the name suggests, is the practice of continuously A/B testing and refining a website to present the best version of a page to their audience members.

The attention-span window on the web is shrinking rapidly. Content giants like AOL, CBS Interactive, The New York Times, and Comcast NBC Universal consistently rank among comScore’s top 50 visited websites, but still struggle to develop a loyal reader base. The quality of your content may be strong, but the competition for eyeballs and engagement is fierce.

Website optimization allows publishers the ability to take immediate action based on feedback gained from analytics. No longer will you need to guess what the optimal headline, copy, or layout of an article should be. You can now test several variations of a page, find the winner, and use the winning variation going forward.

As an example of what to test on your website, we’ll use USA Today to show how a publisher can think about optimizing its user experience for audience engagement, audience growth, and return visits. USA Today is not an Optimizely customer, and these ideas are purely hypothetical. (And we thank USA Today for the inspiration!)


With testing data on hand, publishers have the power to make decisions on copy, content, design, and virtually any other variable that affects a website’s success. Furthermore, a publisher can now target content to very specific visitor attributes, getting readers closer to the personalized experience they all crave and enjoy. The end result is a tailored, highly relevant experience for individual readers: imagine a front page article that surfaces relevant local news to visitors from different states and cities.

These statements aren’t based on conjecture. The numbers speak for themselves. Optimizely’s media customers in the Publishing space see an average lift of 29% in pageviews and 14% in reader engagement.

Optimizely wants to put the ability to ask questions of your audience—through running experiments and tests on your website—in the hands of anyone with a role at a publishing company. Our brand new guide, “The Ultimate Guide to Website Optimization for Media Companies,” provides an in-depth and thorough crash course on A/B testing.

For more information about Optimizely, visit

More in