As perfect proof of its point, a link to an article titled A Catchy Headline The Biggest Draw For News Article Readers induced me to click and read it. The article noted that a new study by Harris Interactive showed that catchy headlines topped interesting visuals as a reason to read an article. This is from the Harris release:
A catchy headline is the top influencer on Americans’ likelihood to read an online or print article in full (54%), though the inclusion of interesting pictures (44%) and interesting data or research (43%) are also strong lures… News junkies are less likely to be lured into reading an article by the presence of an interesting picture (32% junkies, 46% moderate, 41% not interested)… Females (58%) are more likely than males (50%) to be drawn in by a catchy headline, while males (47%) are more likely than females (40%) to be attracted by interesting data or research.
This sounds like great news for headline writers. And the male vs. female divide is great, actionable information for content publishers aiming at a particular demographic, right? Well, maybe not.
If Only We Knew…
Harris is a huge research firm. They employ over 700 people, many of whom no doubt hold impressive credentials in statistics and other relevant disciplines. I have no doubt that the above data, gathered from over 2,300 survey participants, was collected and analyzed using the best practices for such research.
There’s only one problem. This lovely, detailed data is bogus, despite the large sample size and low calculated margin of error.
What’s wrong with this data? It’s simple. Asking people why they do things is terribly unreliable. Our behavior as humans is influenced by many, many factors, most of which aren’t conscious or rational.
Rationalize This
Even when do things for reasons we aren’t aware of or can’t articulate, we are quick to explain why we did them in rational, logical terms. Indeed, neuroscientist Michael Gazzaniga says we all have something called “the interpreter” that incorporates our actions into a logical and consistent framework. Split-brain research as shown that people have a remarkable ability to fabricate explanations for their actions that seem logical but have no basis in reality. (See How “The Interpreter” Screws Up Market Research for more on this topic.)
No Way, Not Me!
Asking someone whether a picture would induce them to read an article is a perfect example – for many of us, our logical mind will say, “Of course I wouldn’t be so shallow as to read an article based on a picture. It’s the quality of the news. And, yes, a well-written headline can draw me in.”
Get The Real Story
If you want to get the real story on the behavior of your customers, readers, etc., don’t rely on self-reported data. While such data can be fine for simple facts, like, “Did you eat breakfast today?” it will rarely answer questions like, “Why do you prefer Grey Goose vodka?”
While neuromarketing studies are one way to get at real feelings, one simple solution is to measure actual behavior. Don’t ask people if they are more likely to click on a link accompanied by a picture – test it! Today, it’s trivially simple to deliver different content versions to web visitors and measure what really does work. Instead of asking whether customers would buy the product if it was 10% cheaper, try out the lower price on a sample of real customers and measure the boost in response.
In short: worry less about what your customers SAY, and more about what they DO.
Have you encountered research data that was statistically accurate but provided misleading cues about your customers’ behavior or intentions? Share it in a comment!
Great points Roger.
I will mention that Eyetrack III (http://www.poynter.org/uncategorized/24963/eyetrack-iii-what-news-websites-look-like-through-readers-eyes/) has pointed out that headlines *do* seem to at least draw a more immediate (and longer lasting) gaze from readers on news sites.
Good point, Gregory. I could have been clearer in the original post – I have nothing against headlines, and consider a well-written headline to be a great inducement to click and read. But, I don’t think asking people “why did you click?” is going to give accurate results in every case. Eye tracking, as you mention, is one way to measure actual behavior vs. relying on self-reporting.
Roger
Great article, it’s so easy to base conclusions on data presented to you but so often the real story is quite diffferent then the obvious conclusion that is ready to be drawn. From my time as a newspaper journalist student I know that both visuals and headlines are incredibly important and when you have the options you should always be using both to attract more clickthroughs.
I agree about visuals AND headlines as being important, Bram. And it’s entirely possible that headlines ARE the most important factor, but not because readers say so.
Roger
Test your customer response in all seasons (spring, summer, winter, fall) and paying close attention to where the product or service lies (hype, advantage, choice, cost, replacement) within its useful market life. Offering customers the iPhone G4 two weeks after the release of the iPhone G5, may be beneficial to some and pointless to others. At Oevae we help our clients research the market, as we’ve found many small / midsize companies have less time to evaluate / test markets than they do managing their products and services. So as Roger makes his point very clear, testing is the real identifier for what your customers do and want.
Head Honcho
Roger, testing whether someone clicks on a link is certainly simple enough, but how do you connect it with segmentation data like gender and “news junkie”?
Gary, that is a complicated question. Or at least it requires a complex answer! In some cases, you may not know. If you are dealing with logged in users, you may have a wealth of data about them. Or, even without a login, you can track user behavior on your site. Other resources may also be able to deliver data about a user based on third party cookies.
Roger
Roger, good catch, you are right on the money.
This is also why it’s a flawed methodology to ask customers how they found a business, giving options (online, referral, yellow pages, radio ad, etc.).
People simply don’t remember accurately. They can say they heard about a company from radio ads, and the company doesn’t even do radio ads.
I see web forms like this a lot, and not only does it create unnecessary friction for website users, but the data can’t even be trusted.
Another good article and like the other writers, I agree that asking people why they do something never gives good results. I think AB split testing is the way to go.
That’s one of the problems with traditional consumer research. Remember New Coke? The Coca Cola company spent years and millions of dollars testing and asking people what they wanted. They then gave them something they said they wanted and all hell broke loose. The vast majority of our decisions are unconscious left brain ones and when asked to give a logical explanation, our right brain takes over. Of course it doesn’t know anything so it tries to come up with a logical explanation.
Absolutely spot-on.
As delicious as your articles are, I was fully prepared to disagree as I clicked over — full-on skeptic mode — because I thought the article was going to state that authors using the catchiest headlines present bogus data (with them). But here, again, you’ve nailed it.
I wrote an article years ago about product research, wherein I stated,
“Don’t ask subscribers — we lie.”
Not intentionally — well, some do sometimes… in order to sway results, i.e., pricing in their favor — but usually, simply because we don’t know.
While not taking away from the value and power of informed deduction, (necessary, for instance, when evaluating split-test results in marketing), I really appreciate the validity of your opinion here — and the recommendation to pay closest attention to what one’s audience DOES… as that often differs from what they speak.
At least my headline got YOUR attention, Harmony!
Roger
This article might interest you : http://boingboing.net/2012/10/02/what-a-dead-fish-can-teach-you.html#more-184176
Old news, but still relevant, Walter: Are Brain Scan Findings Fishy?.
Great read. Our motto has always been, “only pay attention to what you can measure.” This lead us to make more accurate decisions about how to redesign our website. One great “secret” tool we found is crazyegg.com. If you have a chance, take a look at it.
After reading John Caples “Test Adverting Methods”, I wrote a blog on this very subject:
http://op-ed.ca/2012/09/24/the-most-important-overlooked-and-easiest-way-to-get-more-traffic-to-your-blog/
Some human behaviour doesn’t change, Mr. Caples wrote his book 50 years ago 🙂
Hard to argue with Caples!
Roger
I am a big believer in visual marketing and the affect images have on the subconscious mind when it comes to a prospect taking action. I don’t have precise proof as I’ve only tested my theory on my own landing pages. One thing is for sure, colorful, visual call to actions and creative design work to encourage someone to buy a product. I totally agree that asking somewhat what they might do and what they actually do are two different things.
Great article and so true. That’s why, large companies, when doing research around their target market, often use techniques that speak in terms of metaphors rather than just having them answer questions. You can get at a much deeper “true” meaning that way.
The idea of “metaphorically asking” is so interesting. 🙂
Thank you for clarifying that the quality of information when quoting statistical is undeniably important. Self-report is easily the most superficial type of survey. You’re very right to say that it makes a lot more sense to pay more attention to stats collected by comparing actual activity (like clicks or hits.) Great read!
Great article. Totally spot on. The power of a great headline can not be emphasized enough. Every advertising agency, newspaper and marketing department knows this unequivocally.
I can definitely see where you are going with this, Roger.
If I had to answer whether I was more moved by a headline or an image, I am not sure I’d have a good answer.
I suppose hitting all the points would be a good solution.
Asking users to explain an action void of conscious reasoning or decision making results in nothing but b.s. “Well I hadn’t really thought about why I clicked on this article, but the half naked woman in the picture might have had something to do with it” isn’t going to be a very popular answer on your poll. No matter how true it is. Must have been that catchy title…
The only way to find out what people will do is to watch. Figuring out why has been the goal of everyone, from psychologists to marketers to angry parents. Those who do will be those who influence.
Exactly, Ben. People are even less likely to answer honestly if the response might make them look bad. In truth, of course, they often just don’t know.
Roger
Yeah, right. But you completely used a “catchy headline” to make me read this post. And I didn’t find anything about catchy headlines…
That’s because the post wasn’t about headlines. It was about the use of bad data to justify a branding technique of using catchy headlines rather than catchy graphics…
Very True Roger, These day, people just about write anything just because they can get to people to read their articles. This effects the reputation of the bloggers and the businesses.
I was so excited when a big report on why Coloradans donate came out – until I learned about their methodology. It’s exactly as you describe – they used a phone survey!
http://www.coloradononprofits.org/wp-content/uploads/understandinggiving.pdf
Great.
Now I have to resurrect my swipe file of waltzing cat pictures.
Darn you, Roger!
Good point, Gregory. I could have been clearer in the original post – I have nothing against headlines, and consider a well-written headline to be a great inducement to click and read. But, I don’t think asking people “why did you click?” is going to give accurate results in every case. Eye tracking, as you mention, is one way to measure actual behavior vs. relying on self-reporting.
Of course I wouldn’t be so shallow as to read an article based on a picture. It’s the quality of the news. And, yes, a well-written headline can draw me in.”
Exactly, Ben. People are even less likely to answer honestly if the response might make them look bad. In truth, of course, they often just don’t know.