Last fall, the Chartbeat product team was hunkered down in an office space that could’ve made an excellent interrogation room. We temporarily obtained this 500 square-foot room to augment our main office, a sardine can of developers, designers, analysts, scientists, and a growing sales and marketing team.
It was… austere: four brick walls and a cement floor. There was a glass-topped table in the middle, a whiteboard, and a phone. Two windows separated us from the round-the-clock demolition going on in the adjacent lot, and you almost always had to shout to be heard. We even called it the murder room.
We were examining the roadmap for the Chartbeat Publishing dashboard. There was a lot on the table—all kinds of ideas for functionality that we wanted to add to a product that was starting to look like it had too much going on. There was no way we were fitting it all in to the current UI. The bulldozers were looking like a good idea. It was time to start from scratch.
But in reality, prep work for a top-to-bottom overhaul was already well underway. We had initiated a massive effort at capturing the state of the newsroom and the publishing industry, and were already thinking about how to align Chartbeat’s services with those conclusions.
Our Research Effort
Research is an ongoing practice at Chartbeat. We’re constantly talking to our clients, figuring out how they work and why they do what they do – even sketching out ideas together and evaluating concepts. Nevertheless, heading into the project, we wanted to consolidate of all our meeting notes and interviews, and confidently answer the following questions…
-
Who’s using our product?
-
What are their motivations, needs, and philosophies?
-
Where’s the industry going?
-
What will the newsroom look like in a year or two?
-
How will editorial roles evolve?
If we started by simply answering these questions, we knew good things would happen.
Gathering Information
To approach our research challenge systematically, we used ethnographic methodologies:
Interviews and Field Studies
If we weren’t on the phone firing away at our customers with non-leading questions, Mona Chaudhuri and I were hitting our clients’ offices on a semi-daily basis throughout most of 2012—picking brains, hearing war stories, watching them work, and bouncing ideas around.
Copious interview notes came from these many many meetings at places like The Blaze, NBC News, the Wall Street Journal, The New York Times, CNN Money, Fast Company, Slate, Financial Times, and dozens more. If you had an office in New York, one of us was knocking on your door. And if your offices were outside of New York, we were there too: Washington, London, Toronto, Berlin, San Francisco.
Diary Studies
We asked a diverse group of Chartbeat customers to keep journals of their day to day activities. The journals were written over the course of three weeks into Posterous (R.I.P) blogs. Some of the participants were given, i.e. gifted, iPads to more easily facilitate the entry of notes and ideas. Yes they were great as a lightweight field tool for entering notes, but more so the iPads were a great incentive to keep participants motivated.
We had some very prolific contributors… for example this guy: Jonathan Tesser at New York Magazine (at the time). Reading what his day was like in his own words was a fantastic window into newsroom issues. The ups and downs were so much more tangible—you could really feel the personal challenges in a way that other research methods just couldn’t uncover.
Surveys
To get a quantitative perspective on newsroom ethnography, we conducted a survey, which asked people about their role, three day-to-day responsibilities, and the three long-term objectives that they are evaluated on.
Processing the Data
We dug into the survey data and immediately got to some interesting information. For example, 64% of respondents reported themselves as some type of “content creator.” And 36% identified their role as being at least partially on the business side. In our fieldwork, we were still talking primarily to editors and writers, so it was somewhat of a surprise to see that one in three people had some involvement in other aspects of the business, too.
We took the diaries and interview notes and boiled them down, then reduced them, and then reduced them some more into a mental model diagram (shout out to Indi Young and her fantastic book on the subject). The mental model represented everything we knew about newsroom behavior—it contained every discrete action or behavior taken by people in the front lines of a newsroom. There were a lot and they were extremely varied, for example:
-
“curate third-party content on Tumblr”
-
“harass writers to meet their deadlines”
-
“look for dead spots on the homepage”
above: a couple branches of the mental model
above: a grouping of activities within the branch: “Understanding referrer sources”
We consolidated the individual actions—several hundred—into larger groups. For example “curate third-party content on Tumblr” was put into a group called “build off-site brand presence.” And finally, all the groups were assembled into four high-level categories:
-
Developing content
-
Assessing content
-
Assessing audience
-
Developing audience
Everything that we observed and captured fit into one of those four categories. That gave us a way to maintain a broad perspective on the publishing business as a whole, with the means to narrow our focus down to specific workflows and actions through a highly organized affinity diagram.
At this point our ‘forensics’ work was done. Well, it’s never done, but we’d just completed a very thorough and immersive look at newsroom culture, workflow and the state of the publishing industry.
The output of this work – the mental model – gave us something to measure our product against as well. What actions were we supporting and not supporting? We brainstormed all the realistic and totally unrealistic things we could do to help our customers across the many facets of their work.
Tomorrow, in part two of this post we’ll focus on some specific findings of the research and how we used it to roadmap the next incarnation of Chartbeat Publishing.