Choosing journalism metrics that actually count (and are countable)

Stephanie Snyder
Engagement Manager

stephanie.snyder@wearehearken.com

Julia Haslanger
Engagement Technical Account Manager

julia.haslanger@wearehearken.com

Fluffy goals, unmeasurable metrics, and forgotten impacts, oh my! Here’s how we’re helping newsrooms fix all of that.

As Hearken has evolved over the last three years, so has the journalism industry. We’ve gone from helping a handful of newsrooms better listen to their audiences and the public they’re trying to serve, to a roster (that’s still growing) of 140 news organizations. Why? Audience engagement has moved from a supplemental project to a necessary strategy rooted in building public trust and growing reader / listener / viewer loyalty. And people in newsroom leadership — with ties to editorial and business priorities — understand that.

One major stumbling block we discovered was that news organizations didn’t know how to measure the goals they knew (or heard) they should be setting around better connecting to their audiences.

Starting in spring 2017, we began to centralize the goals our partner newsrooms were stating when they started working with us, and noting any metrics they specified or seemed to be using. As we updated and analyzed that document throughout the year, two themes became clear:

• Newsrooms are not practiced in setting measurable goals. (An example of an unmeasurable goal: “We’d like people to feel that they’re being listened to.”)
• The metrics newsrooms picked often didn’t actually map to the goal they stated. (Example: “Be less of a one-way street” paired with the metric of “page views”.)

Another problem emerged over the course of 2017: As we began working with newsrooms, we asked them for editorial goals and business goals. Rarely did the editorial employees have a sense for how the business team could measure success.

In early 2018, Hearken knew more than 20 news organizations would join us over a short period of time through the Community Listening and Engagement Fund. This was the perfect (and necessary) moment for us to create a streamlined process to help newsrooms set goals, identify metrics, and track them. We set out to create a menu for newsrooms, offering them options for goals and a selection of metrics that directly map to measuring their progress toward that goal. The menu has three sections, one for organization-wide goals, one for the journalists’ goals, and one for the business team’s goals.

How we got there

The first step we took was to organize all the goals we’d heard over the past two years, both from our current partners and from people Hearken’s business team talked with during the sales process.

We sorted them into three piles:

• Too big, vague, need nested goals in order to measure (E.g. “Serve people better” or “Build trust”).
• More specific, but still need nested goals in order to measure (E.g. “Create more inclusive journalism where people see themselves reflected”).
• Very specific, measurable (E.g., “Hear from readers in ZIP codes outside our core subscriber base”).

Looking forward: A focus on impact

Part of this process also involved us separating goals from impacts. Goals are the things you want to have happen as a result of doing this work. Impacts are the things that actually happen, which may not be tied to any goals or metrics you set.

Now that we had designed a good system for helping newsrooms think about goals, we turned our attention to helping them think about impacts. The first challenge was helping newsrooms understand how it benefits them to track impact at all. Once we drafted some go-to language around that, we began shaping a system for them to track it.

As we thought about impact in the framework of our audience engagement ring (more on that here), we broke potential impacts into three categories:

Civic engagement
Examples:

• People said they were able to make a decision on a ballot measure with your help.
• More people vote in local elections.
• Increased public knowledge and action around a particular issue as evidenced through community participation in events.

Institutional change
Examples:

• Respond to common themes of questions by establishing a new beat, special series or event.
• Journalists get in the habit of looking to their communities for story ideas and questions, rather than their own experiences or competing media.
• Media outlets and organizations work to hold public officials accountable for and motivate them to fix problems raised by the audience through the public-powered journalism process.

Audience loyalty and brand awareness
Examples:

• A thank you note from a reader whose life was demonstratively improved by knowing the answer to a question.
• People saying they were motivated to join a membership program or subscribe due to your public-powered work.
• Newsroom staff members are invited to speak about and promote the organization’s work and mission.

Just as newsrooms see an increase in community engagement the more they are transparent about involving the public in the editorial process, our hope is that newsrooms will see an increase in impact the more they highlight the change happening because of their public-powered journalism.

Newsrooms can work their way around the pie: Highlighting an increase in civic engagement can help lead to institutional change → highlighting how your community created institutional change with your guidance can help lead to audience loyalty → highlighting your most loyal readers and how they are civically engaged can create FOMO and lead back to more civic engagement.

Now that we have a framework for communicating about impact, our next step will be to draft and test resources that our newsrooms can use to track their own impact. We’ll share an update down the line about what we learn as we develop that impact tracker for (and with) newsrooms. If you have any experience in this space (or ideas) please let us know! We love learning from the community around us.