Editor’s note: This Q&A is a set of follow-up questions to a post written by Membership Puzzle Project researcher Emily Roseman — Loyalty is membership’s North Star. Here’s how news sites & advocacy groups measure it.

Within the post, Roseman spotlights how several organizations (from digital news, to public media, to advocacy) measure their membership program. Ariel Zirulnick, who leads the Membership in News Fund at the Membership Puzzle Project, provided additional insights.

Chartbeat: How are you/the Membership Puzzle Project defining loyalty for the purpose of this research? Should a single, industry-wide definition be adopted?

ER: For this post, I was looking at how a few different organizations (from digital news, to public media, to advocacy groups) define loyalty. I find that loyalty is typically summarized as “regularity,” as in, the frequency the audience uses a product. One thing became very clear across all of these groups in my research: a membership program should focus on building repeat activity and member retention, above all. Together, these two cornerstones make up a good sense of member “loyalty.” 

Chartbeat: When looking across the various media outlets in this research, were you able to identify any key similarities or themes tied to their success in building loyalty?

ER: This question reminds me of the new research by Lenfest and Dot Connector Studio that finds that, when studying engagement tools like Hearken and Groundsource, a commonality across the successful newsrooms was having “a receptive newsroom culture, leadership buy-in, and a dedicated staff members.”

Above all, they found that successful newsrooms often had “engagement superstars.” From what I saw across the newsrooms I spoke with for this post, this also holds up — dedicated staff members and engagement superstars are often the folks who think deeply about the connection between engagement, loyalty and membership. 

Chartbeat: How can single or multi-channel outlets better measure engagement or loyalty due to the unique nature of those audiences? Is segmentation the answer? 

ER: For KQED, Tim Olson describes how they measure loyalty per platform — so for web metrics, for example, they track the number of visitors and the subset of repeat (4x or more a month) visitors. I think that’s a smart way to be sure your organization is tracking loyalty for each platform. Of course, it gets complicated when you have audience members showing their “loyalty” in a combination of ways and on different platforms all at once.

I’d be curious to know if any news or non-new orgs have found ways of measuring an individual audience member’s activity across all platforms and whether there’s a way of assigning a loyalty score or metric that takes into account all of their activity. I know there’s some Google News Consumer Insights tools that can synch with Google Analytics and it gives you a great picture about the top of the funnel, but what about the rest? I’ll be curious to watch projects that center around tool development for publishers like Newspack — a CMS that’s being built for publishers — to see if they can figure this out! 

Chartbeat: You talk about the positive correlation between high value content and greater loyalty. Does your research point to a broader recognition by media of this trend or is there still room to grow?

AZ: Some of MPP’s earliest research looked at early adopters of membership in news, studying the needs and motivations of both the news organizations and their members (organizations studied include De Correspondent, The Guardian, Reveal Center for Investigative Reporting, and the Texas Tribune). One of the primary reasons those members joined is because the journalism they produced offered something different. It was deeper, showed greater integrity, and offered a welcome break from the banter and sound bites that we associate with much of the content produced by news organizations today. You can learn more about that in our membership manifesto, based off of interviews with hundreds of news supporters. 

Chartbeat: Do publishers need to take different tactics when addressing “subscriber” over “membership” models? (i.e., subscriber = transactional, members = involved)

AZ: Yes. A subscription is a transactional relationship – subscribers give their money, and receive a product in return. This works very well for many news organizations, particularly those operating with strong breadth and depth or providing highly specialized information.

“It is a relationship based on passion and trust, and that means the publisher needs to articulate their mission, offer transparency around strategic decisions, and provide opportunities for meaningful participation to their members.”

But with membership, members are often contributing not just their money, but their time, energy, expertise, and ideas to a cause that they believe in. It is a relationship based on passion and trust, and that means the publisher needs to articulate their mission, offer transparency around strategic decisions, and provide opportunities for meaningful participation to their members, among other things. We assessed that consumer vs member difference in research published in April.

More on this topic

Emily Goligoski, former research director at the Membership Puzzle Project, provided insights on the membership versus subscriber divide at Chartbeat’s office earlier this year.

Chartbeat: What can under-resourced organizations do to improve engagement and loyalty in the absence of dedicated audience or growth teams?

ER: The research out of the Shorenstein Center’s Single Subject News Project dives into this question, and found that for many small and scrappy newsrooms, it’s a smart choice to focus time and resources on key products like the email newsletter and old tactics like SEO. Both of these tactics can work to build and engage audiences, and can typically be integrated across the workflows of small newsroom staff. I’d recommend checking out the email newsletter section of the Single Subject Project’s medium blog, and the Email Newsletter Guide (a collaborative project with the Lenfest Institute and Yellowbrim) for more on email best practices for publishers. 

Chartbeat: You hit on something important in noting, “Efforts to track this engagement and subsequent loyalty are currently disjointed and manual.” What do you believe is needed to address this overarching issue?

ER: I think it’s both a technical and workflow issue. On the technical side, I heard frustration from many of my interviewees about lacking a tech platform that allows them to track or tag repeat activity per user across platforms. Again, I think we’ll be watching initiatives like Newspack, Pico, and other tech that’s being built specifically for newsrooms to see if we can have a technical solution to the problem. But even with the “right” tech stack, workflow issues arise. We typically see efforts to track audience engagement fall to roles like Audience Engagement Editors — these bridge roles that sit in between editorial and revenue — and having a bridge role person who is empowered to collect and analyze this data across teams (and then is empowered to make or lead changes based on the data) is incredibly important. 

“…having a bridge role person who is empowered to collect and analyze this data across teams (and then is empowered to make or lead changes based on the data) is incredibly important.” 

AZ: To add to that – Expecting all your audience members to come to your website or read your newsletter is unrealistic and likely leaves out many potential readers. To engage well with community members, publishers need to meet them where they are. But that’s likely many different places: newsletters, Facebook pages and groups (possibly closed ones), Instagram, Reddit, events, and partner organizations, to name just a few. Automated tracking for website and newsletters has become relatively simple, but it’s still a very manual process for these other channels. And while you can see data such as the number of followers, there’s no way to link, for example, commenting activity to a person’s user profile with a publisher. For events, it becomes a question of how much friction you’re willing to introduce into the check-in process.

Yes, you could collect a several types of information from people as they arrive, but that leads to a long check-in line and makes the event feel more formal than you might want it to feel. Post-event e-mail surveys generally have low reply rates. Publishers are constantly assessing how much staff time they can commit to tracking activity manually, how that will affect their relationships to collect that data, and how useful that data is.

Chartbeat: You identify specific metrics, including MeRa (Members Returning for Action) and Regularity, as indicators of loyalty. Why haven’t we seen more media organizations adopt these metrics to solve for engagement/loyalty data fragmentation you point to in the research?

ER: We know that many news organizations have different end-goals from advocacy groups (and metrics should always be tools to reach those end-goals), I do wonder if metrics like MeRa or the “powerful campaigns” metrics from Change.org would translate well into a news environment.

With Change.org’s “powerful campaigns” metric, for example, they are measuring petition signatures.  I know that among many news organizations in the nonprofit space, there’s hesitancy around asking audiences to take actions like this (like to share a story with a friend to show that it matters, or to start a petition) out of fear of being perceived as an advocacy group. These philosophical questions around the differences between advocacy and journalism are much better explained in Dick Tofel’s white paper,“Issues around Impact.”

Chartbeat: What steps can organizations take to improve retention rates, which you point to as a leading indicator of maintaining or growing reader loyalty?

ER: I don’t know. But I think a lot can and should be learned from public media. When I learned about KUER’s retention rate for members (over 12 months, 3 years, and 5 years, it’s a whopping 93.24%. 69.54%, and 57%, respectively), I think I gasped.

“I think that building trust with your audience and repeatedly reminding your audience of your value are two great places to start.” 

Then I compared it to the LA Times, who added 52,000 digital subscriptions over a year, but then were left with a net increase of only 13,000 subscribers in the end. So, public media is doing something right, and I think that building trust with your audience and repeatedly reminding your audience of your value are two great places to start.

Chartbeat: Based on your research, what recommendations/best practices can you suggest for organizations attempting to measure and benchmark loyalty?

ER: I don’t think anyone has figured out a one-size-fits-all approach. It all varies by newsroom —newsroom goals, their audiences, the tools they use, the capacity they have, etc. What seems to matter most is to commit to a culture of testing, which is a common Table Stakes mantra. I think one major best practice is to be aware of the experiments and learnings from other fields.

Where.by, for example, came up with their own back-of-the-napkin solution, which I think is smart and is described in the post. Some public media groups pooled their metrics and came up with that fantastic grid of recommendations on which loyalty metrics to track — see the Current article for more on that. Groups like the Lenfest Institute are working hard to gather benchmarks on what moves the needle for subscriptions. Then we have resources from adjacent field, like the M+R benchmarks report that I point to in the end. 

All of this points to regularity mattering — but it’s up to the newsroom to determine which products work best for their audiences, and then measuring regularity of use for those products.