Global publishers and content creators use our headline tests to better engage their homepage audiences. As the gateway to even more of your hard-earned content, we see the homepage as a massive opportunity, not only for experimentation, but also to increase high quality reader interactions.

We wanted to further explore the connections between headline experimentation and stronger audience engagement as a means to encourage more quality interactions. Therefore, we thought it’d be worth delving deeper into:

• Our analysis of the data behind stronger homepage headlines
• The ways in which headline optimization will help you to answer critical questions about what is needed to engage more readers
• Ways to better position your organization for experimentation

What headline testing data tells us

First, we wanted to answer common questions on the strategies that encourage more engagement with homepage headlines through our analysis of global network data.

Why do we test headlines in the first place?

Our data shows that just by running a test, you can drive more readers to your stories 62% of the time. This begs the question — how much of a difference do alternate headlines make for readers? A significant one — here’s why.

When an alternate version of the headline “wins”, we see an average of a 78% lift in clicks. However, we want to focus on more than just the click to ensure the headline isn’t leading a reader to a content dead end. When we took that into account, we found that headline testing generated 71% more quality clicks. Therefore, heightened levels of engagement aren’t just a boon for your audience engagement efforts — they’re also a major opportunity to build loyalty.

A recap of our recent webinar on headline testing, along with a how-to video on using the tool, is available here.

In Chartbeat, headline testing was built to mirror the realities of managing a homepage. We call out the headlines that need the most help (as you see in the markers below), then let you run a test without having to open another dashboard. This is done through our Heads Up Display (HUD), which is an overlay for your homescreen, section or article pages. The goal is to give you a real-time barometer of your curation efforts to make the small adjustments along the way that increase engagement.

The anatomy of a successful homepage headline

To understand the anatomy of a successful homepage headline, we analyzed the linguistic traits of approximately 100,000 headline tests and 250,000 individual headlines. Overall, we found out that certain words both helped and hurt performance. For instance, interrogatives such as “what” and “where,” as well as numbers, quotations and superlatives (like best and worst) can increase the likelihood of quality clicks. In contrast, using question marks or time references can actually hurt engagement.

One caveat — your audience and content is unique. While audience data can inform better headline writing, the metrics are there to supplement your news judgement, not replace it. Therefore, we recommend using these findings as a starting point for your own experimentation.

Putting these tips to the (headline) test 

So how does this network data apply to the success of headline tests? The first step to informed experimentation is identifying headlines you want to test. First, it’s important to select the stories that will not only have the most impact, but also make the most of your limited bandwidth. Now it’s time to experiment. Here are a few key experimentation tips: 

1. Test in high-traffic areas

You will see the most impact by testing headlines that will be seen by most readers. Testing 1,600 pixels (i.e., under digital the fold) down your homepage isn’t going to get you where you need to go. That homepage optimization is even more critical when you consider the growing number of readers coming directly from mobile devices.

2. Optimize your underperformers

This is a tactic you can take without Chartbeat, as your analytics can point to overall underperformers, which may be due to a disconnect between your readers and headlines. If you have Chartbeat, our Heads Up Display can help you know exactly where there’s a disconnect between headlines and readers. 

3. Spend extra time on your high-value journalism

It’s especially important to fix that disconnect on your high-impact journalism. Breaking news tends to drive more clicks. However, deeper, investigative journalism pieces can be more difficult. Since you spend a lot of time producing these stories, it’s worth making sure you spend equal parts experimenting to uncover the most engaging headline.

See headline testing in action — receive a demo of Chartbeat.

Here are some additional tips for Chartbeat users once you’ve determined where to test:

1. Test regularly, gain actionable insights

Since there’s no limit on how many experiments you can run at the same time with our headline testing tool, you always have the opportunity to learn from your audience’s behavior. Our research shows that the winning headline in a five variant experiment typically has more than a 50% higher CTR than the average headline, whereas you may only see a 23% benefit for a standard A/B test. That means the more options you test, the better.

2. Don’t stop tests prematurely

It might be tempting to stop a test when one headline quickly takes the lead, but you’re likely to see cases where the tides shift halfway through the experiment and the initial success trails off. Let the test run its course since the results can be as diverse as your readership.

3. Ties can point to headline commonalities 

It’s possible that two great headlines are attracting similar levels of engagement. If you’re frequently seeing similar results, it might be that the headlines you’re testing are too similar.

Quick tip for users: You can see your headline test results from the Heads Up Display by selecting ‘Headline Testing Results’ at the bottom of the dock, from the Heads Up Display pop-out menu, or select ‘Headline Testing’ under the Optimization tab from the left-hand navigation. For technical support, see our headline testing guide.

4. Pay close attention to your testing audiences

Our data tells us that homepage audiences are usually loyal, visit often, and know what to expect, whereas social visitors tend to be new and interested in the latest information. It’s likely they’ll prefer different headlines – consider your headline strategy accordingly.

5. Keep track of effective tactics

It is important to monitor what is working. Use our findings as a starting point for your own experimentation and keep track of your headlines that lead to real engagement to see if these findings hold true for your audience.

Chartbeat users can also run a deeper analysis on your test results by exporting the data using .CSV files, as shown here:

Tests.csv offers insights into the optimal number of variants for your site and the users or sections that are running the most/best tests.

Headlines.csv gives you a breakdown of how every headline variant performed within every test that you ran. Think of this option as a way to dive into characteristics of winning headlines like character count and language.

Headline testing is only as effective as its usage

We’ve often seen that headline testing is only as effective as the amount it’s used by your organization. When organizations have worked together on headline testing, we’ve not only seen improvement across engagement data, but their collaborative culture.

We’ve outlined a few of those stories, including how Utah’s Deseret News gamified headline testing (shown above) to see a 45% average clickthrough rate increase. You can see more of those stories on our blog.

Running a successful headline testing strategy: Our takeaways

Our data has shown us that headline testing is one of the most effective ways to solidify your homepage as a key gateway to deeper reading by your audiences. Here’s some ways you can get started:

1. Identify areas of need

Make the most of your time by identifying the headlines in most need of your expertise. For Chartbeat users, the Heads Up Display was designed to give you your data right on your home and article page where it’s most actionable.

2. Know your audience 

Once you identify those headlines, it’s worth considering that your writing tactics for one type of content may not be a one-size-fits-all approach. This is why it’s important to test often so that there is a larger dataset that can help inform your long-term headline strategies.

3. Test to success

It’s difficult to see the impact of your experimentation until it becomes a part of your workflow. Your team knows your audience best, so make sure that experimentation does not come at the expense of your better judgement.

4. Collaboration is key 

We’ve seen the benefits of everything from brainstorming with colleagues from different departments and sections to gamifying the process to discourage a slowdown in testing or inertia.

Subscribe to our monthly newsletter to receive our latest research and product announcements.