wayne120Recently, I came across an interesting Bloomberg Businessweek article discussing the limitations associated with focus groups and the ways some companies were using solutions (such as Invoke’s large-scale online focus groups…ahem, I work there) to overcome these limitations. The article is here: (http://www.businessweek.com/stories/2005-11-13/shoot-the-focus-group).

The article starts out with Yahoo!’s then-CMO Cammie Dunaway talking about killing focus groups at Yahoo! It then goes on to talk about the growing exasperation with focus groups and points out some of the well-known limitations of focus groups, citing some real-world examples from AOL and Pepsi of focus groups gone wrong. Specifically, the author David Kiley spends a good deal of time talking about the false positives that often arise as a result of what he calls the “peer pressure of focus groups.”

One of the more interesting parts of this article? It was written in 2005! 9 years ago, people were exasperated with in-person focus groups. So what’s the state of in-person focus groups these days then? Eh, pretty much the same.

I have spent a good deal of time at Invoke talking about the limitations of traditional focus groups. In fact, I recently gave a presentation at a New England Market Research Association (NEMRA) event that  talked about online large-scale focus groups (ahem…such as Invoke’s) as the next step in the evolution of market research. And in that presentation, I spent a good deal of time talking about the limitations of traditional focus groups, such as:

  • Inherent bias – This is what David is talking about when he mentions “false positives” and “peer pressure.” Groupthink is a good example of this – when members of a group strive for consensus and set aside their own beliefs or attitudes to achieve this. Dominant voice is another example – when one member of a group dominates the conversation and cripples the open, honest conversation focus groups are supposed to deliver.
  • Lack of consensus/open to interpretation – Also, due to these small sample sizes, researchers and stakeholders are often left focusing on 1 or 2 respondents rather than a majority. This creates a lack of consensus around the data as stakeholders may each be focusing on different respondents in the room. I have been in many situations where I have seen stakeholders each walk out of the focus group with different takeaways. This can create tension and disagreement, even when the final results roll in.
  • Small sample sizes – Typically, in focus groups, we are listening to a relatively small group of people voice their responses (between 8-10 per group so even with three groups, we are talking like 30 people). Basing decisions and actions on such a small group of people is a risky endeavor.
  • Time consuming and expensive to do it right– Properly running a focus groups requires running multiple sessions across multiple geographies. This creates a good deal of expense (not only research costs, but travel costs as well) and adds time to a project. And this is before you even get to analyzing this data, which can be a bear.

Rather than “shooting the focus group, maybe we should think of going after focus groups as “Attacking the Dinosaur.”  The term “dinosaur” is such an apropos term to describe traditional, in-person focus groups. Focus groups came about because researchers wanted to gather open, honest feedback from participants. However, as noted above, this can be compromised by the issues that come from placing human beings in a room together. This notable limitation, along with the other challenges listed above, should have researchers questioning why there is still such a strong attachment to traditional focus groups. There are better ways out there. The connective power and anonymity of the Internet along with other technologies that allow us to interpret and analyze qualitative data have given us new ways of talking to consumers.

While I don’t see online focus groups as becoming totally extinct, I have to think them approaching endangered status. And, call me biased if you want, but I think large-scale online focus groups could be the meteor that does it.

RAWR,
Wayne

10 Comments

  • Christine says:

    In your synopsis you see the online focus group as an equivalent to the typical 6-8 people in a room focus group and the alternative to be: “large-scale online focus group”–is that correct?
    How do you define “large-scale”?

    • Wayne says:

      HI Christine,

      Thanks for the comment. Yes, that is correct. I see large-scale focus groups as answering a lot of the challenges that currently come hand-in-hand with traditional groups.

      By large-scale, I mean groups that can range anywhere from 50 people at one time to up to 300. Obviously, making sense of up to 300 open-end responses in real-time can prove challenging without the right tools. But large-scale focus groups (such as Invoke’s) use a few different tools (such as keyword coding and data filtering) to make sense of this data.

      Thanks,
      Wayne

  • fiona ray says:

    Interesting perspective Wayne, I would venture to say that there is a huge array of ‘bad’ qualitative out there, just is there is ‘bad’ quantitative. Typically this is because the method was not in line with the research objectives. Using focus groups to make a decision on which way to go (e.g. will this new product succeed? will this advertising drive sales?) will likely result in ill-formed decision making. Qualitative research ultimately is about hypothesis generation, not validation.

    • Wayne says:

      Hi Fiona,

      Thanks for commenting.

      I agree – there is bad research all around us. But I don’t see why we can’t start thinking about things differently. True, traditional groups cannot be used to make a decision like you describe above. But, why should that stop us, as innovators, from thinking that maybe there is a way to do so.

      Our method doesn’t help with forecasting or projections but it does carry a large enough base size to say, for example, that X% do not like a certain concept or advertisement so perhaps we should drop it. Or that certain other concepts rise to the top. We do qual/quant, which in my opinion cannot be compared back to traditional groups. It’s something totally different and here, in these groups, we can do both.

  • JAN says:

    Focus groups are very much like Bruce Willis in 6th sense: A walking dead. As Willis in that film, fg’s will still be active for some time, giving good insights to some special people for some special reasons…

    The issue is not how fg’s have been useful in the past or if they have the right to exist. It’s just that they are less viable every day. As qualitative researchers we have to evolve into new methods and uses (eg. Microfocused hangouts, deep filling the blanks, etc.

  • Margot says:

    Thankyou for a thought provoking article. I see many of your points and obviously agree on the limitations of trad focus groups but I do not dismiss them as dinosaurs. We in the industry understand these limitations, as do switched on clients, but they also understand their value.
    Similarly most other forms of research have their limitations including large scale online qual groups. The thing is that it is up to the skills of the researcher to make sense of the answers and insights we get.
    These are exciting times and we have never had so many tools to deliver intelligent consumer insights to clients. It is up to us to use the correct tools for the job.

    • Wayne says:

      Thanks for the comment Margot…

      My whole “endangered/dinosaur” comparison to focus groups is more to say that I think new methodologies like large scale online groups could be threatening as a bid-winner for traditional groups in situations where groups would have been the de facto choice.

      I think because online LSFGs overcome many limitations of traditional groups and bring their own values to the table, that in many instances (not all) I do think they would be the better choice over traditional groups.

      I agree – these are exciting times because of the wealth of tools and methodologies available to us. And being in the middle of it, I feel like we are at a critical step in the evolution of market research.

  • Phyllis says:

    what do you mean by your sign-off: rawr?

Leave a Reply

Your email address will not be published.