When you publish a story, how do you know if it is "successful"? What do you want to happen? What impact do you want your work to have on the communities you serve?
At the International Center for Journalists, we hear these questions often from journalists from around the world. They share a common thread: a desire to spark conversations and reactions among people, communities and officials in government and other institutions.
Unfortunately, it's hard to see and measure conversations when they happen. What we can measure easily is the indirect evidence of attention and engagement, through website metrics and social media reactions and shares.
Traffic and engagement metrics tell us that readers were present, but not whether they actually read or understood the story. Similarly, data about "social conversations" only hint at the real conversations that a story may have inspired.
Moving past the numbers
To understand journalism’s impact beyond the numbers, we need ways of capturing more intangible evidence — proof that readers and viewers have absorbed information and were inspired to share it. The Center for Investigative Reporting (CIR)’s Impact Tracker pioneered a practical approach to capturing information about this qualitative evidence of impact, providing a framework to record and classify conversations, events and other actions and reactions. This project inspired ICFJ to develop its own impact-tracking database using Airtable, a cloud-based platform that combines features of spreadsheets and more complex databases.
ICFJ impact tracker in Airtable, showing descriptions of impact events and related published stories. Events are categorized by impact type to facilitate further analysis.
In our Airtable database, we record "events" related to the journalism we support, and categorize them by the type of impact they have. Following the model of the CIR Tracker, the list of types we use includes impact at the individual and group level, in addition to broader effects on government and institutions. For each event, we include links to specific published stories so we can maintain a connection between the actual product and the events surrounding it.
Tracking the conversation
In addition to tracking specific impact events related to the work we support, we're also exploring ways to discover evidence of how that work influences public discourse and decision-making on a broader level. The Media Cloud platform, developed by the MIT Center for Civic Media, makes it possible to view how and how often international media sites cover specific subjects. Before starting the GenderAnd reporting project in India, we used this tool to profile the scope of gender coverage in Indian media and the ways in which those outlets reported on it. As the project progresses, we hope to use MediaCloud to find examples of how this work has changed coverage of gender-related topics. Our newest ICFJ Knight Fellow, Pedro Burgos in Brazil, is developing the Impacto platform to search for evidence of a story's impact on the broader media conversation as well as government decision-making. This is another promising venture that has the potential to enable journalists and newsrooms to understand how their work affects their society in a more direct way than traffic and social media buzz.
As revealed in ICFJ’s recent survey on the State of Technology in Global Newsrooms, most newsrooms are still at an early stage of development in using metrics and other tools to measure their work’s impact and reach. If you're looking to advance to the next level, the concepts and tools mentioned here can serve as a starting point. While it may be true that pageviews still matter, these tools and approaches offer an opportunity to move beyond eyeballs and shares toward a more holistic understanding of your journalism’s impact.
Main image CC-licensed by Flickr via Tomasz Pro.