This post originally appeared on the Tow Center for Digital Journalism Blog on October 30th, 2013.
Journalists are seeing an explosion of quantitative data about how readers interact with what they write. To date, much of the conversation about metrics and news has focused on the dangers of using metrics to guide news judgment or, on the other hand, the risks of ignoring metrics completely. But crucial empirical questions about how metrics are produced and put to use remain largely unanswered. How do analytics firms measure complex qualities like engagement, make predictions about the future performance of content, and communicate with journalists about the value of metrics? And how do journalists at different types of news organizations use analytics in their day-to-day work? Are increasingly sophisticated measures of stories’ performance shaping journalists’ ideas about what is important, interesting, or newsworthy? Has the availability of such data changed the internal dynamics of news organizations?
These are some of the questions I aim to tackle in my Tow project, using qualitative methods, such as ethnographic observation and in-depth interviews, to better understand the development and use of metrics in an analytics firm and two news organizations. But before I can answer these questions, I have to ask a different one – one that is as dreaded in my field (sociology) as it is common: What is this a case of? Even though researchers have a tendency to become infatuated with the most minute details of our subjects, what we’re ultimately trying to do is identify and account for patterns in the social world. The classic “what is this a case of?” question prods us to zoom out, to put things in context, to consider the broader implications of whatever it is we’re studying.
So, what are metrics a case of? While there is a burgeoning movement to measure journalism – especially non-profit and investigative pieces – more qualitatively, most metrics categorize and count – page views, unique visitors, time on page, drop-off rate – with the aim of comparing things: pieces of content, news organizations, authors, readers. Metrics, and the big data trend of which they are a part, represent what philosopher Ian Hacking calls “an avalanche of numbers” made possible by astonishing advances in the ability of computers to collect, store, and process huge amounts of data. And metrics aren’t the only numbers in the avalanche – more and more journalists are now deriving stories from their analysis of quantitative datasets.
Sociologists Wendy Espeland and Mitchell Stevens have argued that quantification, like speech, is “a social action that…can have many purposes and meanings” that arise and shift through use. Scholars who study quantification have sought to uncover these purposes, meanings, and uses: here are two ideas from their research that can guide our thinking about the role of numbers in the production of contemporary news. The first concerns metrics, the second concerns the growing prevalence of data journalism.
- Numbers can discipline, even in cases where they aren’t intended to. Michel Foucault famously argued that statistical measures have been used in attempts to control and “normalize” populations that were considered deviant. But sometimes numbers that were meant merely to measure inadvertently serve a disciplining function. U.S. News and World Report’s law school rankings were designed to provide prospective students better information about schools they were considering, but Wendy Espeland and Michael Sauder discovered that law school personnel internalized these measures, and began to change their admissions and financial aid policies to better conform to them. In other cases, however, the implementation of quantitative accountability measures faces considerable resistance and resentment, such as in Tim Hallett’s ethnographic study of the faculty at an urban elementary school. My preliminary research (as well as work by C.W. Anderson, Pablo Boczkowski, and others) suggests that both internalization and resistance are present in journalists’ response to metrics.
- Numbers can establish trust where it is lacking. Historian Theodore Porter has argued that because of numbers’ longstanding association with rationality and objectivity, quantification can be a useful “strategy for overcoming…distrust,” especially in professional fields that are susceptible to outside criticism. At a time when public trust in journalism has dropped precipitously, then, we might expect the standards of journalistic evidence to become increasingly quantitative. As Tim Berners-Lee puts it in The Data Journalism Handbook, “it used to be that you would get stories by chatting to people in bars, and it still might be that you’ll do it that way sometimes. But now it’s also going to be about poring over data and equipping yourself with the tools to analyze it and picking out what’s interesting.”
Contemplating the increasingly important role of quantitative data in journalism also leads us to interesting questions about the idea of “objectivity,” and specifically about the relationship between scientific and journalistic definitions of this term: How do they overlap? Where do they conflict? If we are indeed seeing a quantitative turn in journalism, will it push these two conceptions of objectivity to be reconciled? These are questions I’ll address in future posts.
Caitlin Petre is a Tow Fellow working on a project on Metrics:Production and Consumption for the Tow Center for Digital Journalism. The Metrics:Production and Consumption project is made possible by generous funding from both The Tow Foundation and the John S. and James L. Knight Foundation. To learn more about the Tow Center Fellowship Program, please contact the Tow Center’s Research Director Taylor Owen:taylor.owen@columbia.edu