For a long time, editors and writers decided on their own what was interesting and important, and wrote stories based on their own instincts and experiences. Sometimes that worked. Sometimes it didn’t. Their only real feedback came in the relatively crude form of letters and calls (complaints, usually) and subscription renewals or cancellations.

With the arrival of the Internet came the attempt to measure readership more closely, in the form of the number of readers for a particular story, or visitors to a particular page, with perhaps some information about where the readers came from and where they went afterwards. Social media metrics — likes and retweets — also helped.

But measuring clicks and traffic is a crazy-making way to assign and write stories, or to value journalism. It’s also a rough way to run a business. And so, more and more, it seems, news outlets are trying to figure out how much time readers are spending on which stories, and even where they’re getting distracted and clicking to something else.

One of the most well-known ways of measuring reader engagement in this relatively new ways comes via Chartbeat, a software package that allows subscribers to tell how long each story is being read and even where in a piece readers drop off.

Some outlets have Chartbeat displayed on the newsroom media wall for all to see.

At the national Education Writers Association conference earlier this year, MaryJo Madda, senior editor at EdSurge, didn’t find lots of outlets that seemed to be using it. Education outlets seem to be joining the move towards using this kind of tool.

“I look at it periodically through the day,” says Inside Higher Ed’s Scott Jaschik.  “It tells you instantly who is on which story when.”

The software was named as one of The 10 Most-Used Tools in Today’s Newsrooms in a recent Thoughts On Journalism piece that claimed Chartbeat “has become a lifeblood for many news organizations.”

The information that’s provided – in something close to “real time” – gives reporters and editors helpful information about what readers think about their work. It can also help them decide what stories to write, how to write them, and – perhaps most controversially – how to convince advertisers and funders to pay for what they’re doing.

Reporters are taking notice. “A huge plus of the system is that it tracks more than clicks, but also engagement time,” says Chattanooga Times Free Press staff writer Kendi Rainwater. “It’s helpful to know which stories are being read and which ones get a click for the headline and lede.”

“Engagement time is as important as the number of hits,” said one education reporter whose newsroom displays Chartbeat stats on the wall. “If you can get beyond a minute and a half to two minutes, that is considered very good.”

A Chartbeat dashboard showing engagement time, recirculation, and readers.
FIGURING OUT WHAT WORKS

EdSurge is one of the outlets that’s decided on Chartbeat.  “It is expensive but features offered are pretty unique,” says Edsurge‘s MaryJo Madda. “The micro information it gives us has been pretty helpful figuring out what content seems to be resonating.”

For example, EdSurge has been trying to figure out whether “media assets” (pictures, videos, graphics) really make a difference in reader engagement. (They do.)

And, in contrast to social media analytics, which measure reader exposure, “focusing on engagement metrics is the next logical step for a lot of news organizations,” says Madda. It’s less fleeting, she says, and feels more substantial.However, it can take a while to learn how to use the tools, according to Madda, and “if you don’t have a plan for how you’re going to use, you’re going to get lost.”

ADDICTIVE

“It is addictive — not Pokemon level, but mesmerizing,” says Modesto Bee (CA) education reporter Nan Austin. “At this moment  I have two stories in the top 15.  One has 11 people currently reading it with an average engaged time of 31 seconds. Total reading time of 490 minutes, with traffic driven to it by social media currently up 24 percent – top referrer is Facebook.”

According to a 2014 TIME piece written by one of the founders of Chartbeat, the average reader lasts about 15 seconds.

“I guess I’m glad to have a real-time metric of what folks are reading to make sure my stories reach the wider audience,” says Austin. “But here is what grates: Top web traffic stories from yesterday:  Pedestrian killed on the highway (in top 20 list in three places); local woman mauled by family pit bull; teens arrested in auto thefts; a homeless story; social media traffic on Bush swaying at funeral …. and on it goes … Number 14 was a blog I did, way high for an education story…

“Thoughtful stories have yet to attract the numbers that blood, guts and celebrity gush does on the Web. Chartbeat brings that home in real time.”

In this regard, Chartbeat can demoralize serious journalists just like regular traffic statistics have done for years. “Sometimes it’s good to see strong local reporting do well, and our staff uses it to try to tweak headlines, display, etc., to make sure we’re getting the best possible exposure for good work,” says education reporter Ann Doss Helms of The Charlotte Observer. “On the other hand, a lot of days the most-viewed story is some fluff or freak show item from an outside source.”

The media wall at the Charlotte Observer. Courtesy Ann Doss Helms.

“PLAIN-ENGLISH” REPORTS

Kris Kerzman, who works at a consortium of newspapers and TV outlets called Forum Communications based in Fargo, North Dakota, just started using it about a month ago. Among other things, Forum Communications is trying Chartbeat to help figure out what readers do in between clicks, rather than just knowing how many times and what they click.

Chartbeat a big improvement over Real Time Google Analytics, Kerzman says, in part because the feedback it provides is “a little bit more human-readable.”  The reports Chartbeat provides are in plain English and point out things that “might not be readily apparent if you’re staring at charts.”

“Chartbeat makes it easier for newsroom to have conversations about what’s a good piece of content, where do we look next for a story,” says Kerzman.

HUMBLING FEEDBACK

In contrast to Google Analytics, Chartbeat shows how readers are engaged with the pieces – not just how long. A heads-up display that’s integrated with the browser shows editors where readers generally stop reading. “Most of your readers stopped reading here,” it tells you.

“It’s a little humbling,” says Kerzman. But it can help. Using this information, Kerzman optimizes content (adding links, images, or video) to help stories read better for readers. “Maybe we can get them to read a little bit longer,” he says. “Maybe we can catch some more eyes.”

In a December 2015 roundup of most engaging stories, the only education-related piece to make Chartbeat’s list was The Coddling Of The American Mind.

IMPACT

“I like knowing that people are spending a little bit of extra time that may not be heavily trafficked,” says Kerzman. But the information hasn’t made a giant difference so far. “There are moments where I think, wow, I don’t know that i would have made this decision without Chartbeat. But there are just as many times I would have made the same decision.”

For example, a recent North Dakota story about a referendum on tax hike for school district “wasn’t a super-hot story, but it a was very important one,” says Kerzman. With Chartbeat, he was able to tell editors that the story wasn’t one of top ten pieces of content traffic wise, but “people spent a lot of time with it.”

CAREFUL USE

Asked about Chartbeat’s role in making editorial decisions, Inside Higher Ed’s Scott Jaschik says “it influences but doesn’t dictate.”

“There are very important issue sin higher education that may not get a ton of traffic. I’m not going to stop covering them because Chartbeat tells me they’re not popular. But it would be foolish not to pay attention to what readers want and respond to.”

For example, when Sweet Briar College nearly closed last year, Jaschik says he initially wasn’t sure how interested people would be. It was a small college. “I wasn’t sure how big a deal it would be.” However the reader response was “massive, and instant.” Chartbeat “told me something I might not have been sure of as instantly.”

Alexander Russo

Alexander Russo is a freelance education writer who has created several long-running blogs such as the national news site This Week In Education, District 299 (about Chicago schools), and LA School Report. He can be reached on Twitter at @alexanderrusso, on Facebook, or directly at alexanderrusso@gmail.com.