Ali A. Tehrani is a student in my Multiplatform Design & Storytelling class. For his Master’s project, a requirement for graduation, Ali has studied headline testing and the preference of readers when it comes to headlines that seduce them into reading stories.

Regardless of what generation of journalist you are in, we have always been taught that the fate of our stories depends, greatly, on the headline that heads it. Many great stories have been ignored under a headline that was not enticing. And, of course, there are wonderfully written, seductive headlines that accompany lackluster stories.

In the digital age, however, attention span is shorter —much shorter, indeed, about 8 seconds to react. Those headlines must be sharp and guide us to the story quickly.

Ali’s project tells us that:

“When given the choice between headline A- “Five decades after Kerner Report, representation remains an issue in media”–or headline B- “People of color think whites more positively portrayed in media: study,” readers clearly communicated their preference. In less than one hour, or 57 minutes to be precise, the winner was announced. Headline B engaged homepage readers of Columbia Journalism Review 81% more, in that a bigger percentage of people who saw it clicked on it.”

Ali’s thesis deals with what he refers to A/B Testing, or split-run, a practice that he writes The New York Times started using to test its headlines in late 2015.

The testing consists of using two versions of the same story compared and the one that attracts more attention is the winner. These are always controlled experiments.

Here is an example of another two sets of headlines and how readers reacted to them:

These two headlines were tested in 2016:

A—“Afghan Soldiers Declared Dead, Then Alive and in Debt”

B —“Found Alive, Men Are Told: You Owe Us for Burials”.

You guessed right if you think that the B headline was the winner, by close to 3000 percent.

Fascinated by this information, Ali set out to do his own tests with A/B testing. He surveyed 541 people via Amazon Mechanical Turk, a platform for crowdsourcing tasks which requires human responses, in exchange for compensation.

In the student’s survey, the B headline, “Found Alive, Men Are Told: You Owe Us for Burials”, also won by a 10% margin.

But, here is the interesting fact. When asked which headline the subject would identify coming from a reputable news source, the “winning” headline lost to its counterpart by a 10% margin.

Finally, a toss up between these two headlines for the same story:

A– “Obama urges a Discussion of Race in Cuba”

B– “Cuba Says It Has Solved Racism. Obama isn’t so sure”

Again, the B headline is the preferred, the one that entices readers to read the story, but A is the choice for coming from a more reliable source.

Conclusion: sometimes the seductive headline does not get credit as coming from a reliable source. Do we equate the more boring, textbook like headline with serious and not the one that got us to read the story?

In the era of fake news, this is something to think about.