Joe at American Copywriter put up an excellent post today on focus groups and why they ought to be outlawed.
Mention the words “focus group”, and every creative can feel the bile rising in the throat. We know what’s coming. We’ll be sitting behind a two-way mirror for a couple of hours, eating bad snacks, and watching a group of people casually trample the work that we’ve poured our very souls into. Once those plumbers and sales clerks take their places on the other side of the glass, they suddenly become advertising experts. And their opinions suddenly matter more than those of us who’ve been in the business for a long time.
Joe goes on to explain how famous author and New Yorker writer, Malcolm Gladwell, put the spank on focus groups recently at the AAAA Account Planing Conference in Chicago.
He launched into a one-hour rant on why focus groups lead companies toward bad decisions. His three primary points were:
1) Focus groups are biased to favor the known over the unknown, the familiar over the unfamiliar. Because creative ideas are new and unfamiliar, they will always test badly in focus groups.
2) Asking people to explain their feelings is asking them to explain a visceral reaction – something that happens in the subconscious.
3) Asking people to explain their feelings can actually change their feelings.
Gladwell also gave examples of wildly successful products that focus groups hated–Herman Miller’s Aeron chair, “The Mary Tyler Moore Show” and “All In the Family.” See point #1.
I certainly share Joe’s loathing for focus groups. But when you think about it, advertising isn’t created for “advertising experts” (assuming that’s not a contradiciton in terms). While you can’t blindly follow the dictates of “plumbers and sales clerks,” neither can you dismiss them as know-nothing proles. At the end of the day, it is their verdict that matters – not that of a rarified circle of creatives – because they render their judgement in the most important arena of all: the marketplace.
Great point, Carl. Consumers preferences and behavior are what it’s all about.
But a focus group is not the marketplace. It’s a totally artificial envrironment.
If the MBA-toting brand managers want to test something, let them run one creative execution against another (like they do with direct mail). Then, and only then, will the data collected be worth something.
This is from a Gladwell interview in the New Yorker. The original link has expired, so I am copying this from my own blog entry:
1. The Triangle problem: The Triangle problem shows the instability of preference – start with two samples (Coke, Pepsi) and pick which one is one. People can do this correctly about 80% of the time. However, if you give people 3 samples of two drinks and ask them to determine which two are alike it is significantly more difficult – in fact people can do this correctly 33% of the time which is the same as randomly guessing. Why believe people when they say they don’t like something if preference can be this unstable?
2. The Story Telling Problem: Many times when people tell you why they like things they are just making up stories. Gladwell cites an experiment where top tennis players were asked how they hit a top-spin forehand serve. Without exception, they all said that they rolled their wrist at the moment of impact. However when analyzed with a high speed camera, none of them actually did this. In fact, if you roll your wrist at the moment of impact it’s very difficult to hit the serve. Despite the fact that all of the people they asked are masters at hitting this type of serve, when asked how they do it they are incapable of introspecting what they do and resort to telling a story (most likely how they were told to do it).
3. Perils of Introspection: The Act of getting someone in a room and asking them to explain their preference for something can affect the answer that they are likely to give you. Gladwell cites an experiment where two groups of people were asked to select a poster from a set. However the second set of people were asked to explain why they picked their poster. After 6 months, the people were asked how they liked the poster they had picked. Overwhelmingly, the people who were asked to explain their selection were unhappy with the poster whereas the first group of people were very happy with their selection. Moreover, the first group of people were more likely to pick impressionist painters whereas the group that had to explain their selection were more likely to pick kitten posters. In this case, forcing people to explain why they liked a poster forced them to like a poster that they could easily explain why they liked it. This is a pretty big problem if you are trying to understand why people like one product over another
“If Van Gogh had a focus group, he might have two ears right now.” – Jim Carrey
At the risk of using sounding like the NRA defending the right to wield an Uzi in the office (