Recently, I came across an interesting Bloomberg Businessweek article discussing the limitations associated with focus groups and the ways some companies were using solutions (such as Invoke’s large-scale online focus groups…ahem, I work there) to overcome these limitations. The article is here: (http://www.businessweek.com/stories/2005-11-13/shoot-the-focus-group).
The article starts out with Yahoo!’s then-CMO Cammie Dunaway talking about killing focus groups at Yahoo! It then goes on to talk about the growing exasperation with focus groups and points out some of the well-known limitations of focus groups, citing some real-world examples from AOL and Pepsi of focus groups gone wrong. Specifically, the author David Kiley spends a good deal of time talking about the false positives that often arise as a result of what he calls the “peer pressure of focus groups.”
One of the more interesting parts of this article? It was written in 2005! 9 years ago, people were exasperated with in-person focus groups. So what’s the state of in-person focus groups these days then? Eh, pretty much the same.
I have spent a good deal of time at Invoke talking about the limitations of traditional focus groups. In fact, I recently gave a presentation at a New England Market Research Association (NEMRA) event that talked about online large-scale focus groups (ahem…such as Invoke’s) as the next step in the evolution of market research. And in that presentation, I spent a good deal of time talking about the limitations of traditional focus groups, such as:
- Inherent bias – This is what David is talking about when he mentions “false positives” and “peer pressure.” Groupthink is a good example of this – when members of a group strive for consensus and set aside their own beliefs or attitudes to achieve this. Dominant voice is another example – when one member of a group dominates the conversation and cripples the open, honest conversation focus groups are supposed to deliver.
- Lack of consensus/open to interpretation – Also, due to these small sample sizes, researchers and stakeholders are often left focusing on 1 or 2 respondents rather than a majority. This creates a lack of consensus around the data as stakeholders may each be focusing on different respondents in the room. I have been in many situations where I have seen stakeholders each walk out of the focus group with different takeaways. This can create tension and disagreement, even when the final results roll in.
- Small sample sizes – Typically, in focus groups, we are listening to a relatively small group of people voice their responses (between 8-10 per group so even with three groups, we are talking like 30 people). Basing decisions and actions on such a small group of people is a risky endeavor.
- Time consuming and expensive to do it right– Properly running a focus groups requires running multiple sessions across multiple geographies. This creates a good deal of expense (not only research costs, but travel costs as well) and adds time to a project. And this is before you even get to analyzing this data, which can be a bear.
Rather than “shooting the focus group, maybe we should think of going after focus groups as “Attacking the Dinosaur.” The term “dinosaur” is such an apropos term to describe traditional, in-person focus groups. Focus groups came about because researchers wanted to gather open, honest feedback from participants. However, as noted above, this can be compromised by the issues that come from placing human beings in a room together. This notable limitation, along with the other challenges listed above, should have researchers questioning why there is still such a strong attachment to traditional focus groups. There are better ways out there. The connective power and anonymity of the Internet along with other technologies that allow us to interpret and analyze qualitative data have given us new ways of talking to consumers.
While I don’t see online focus groups as becoming totally extinct, I have to think them approaching endangered status. And, call me biased if you want, but I think large-scale online focus groups could be the meteor that does it.