In-Person Qualitative Research
For decades, an in-person setting for conducting qualitative research has been seen as the optimal condition for the most in-depth method of collecting data. By being present at the time data are collected, a skilled moderator can listen to and understand the direct answers that participants give to the moderator’s questions and combine those data with visual cues from non-verbal behavior such as body language and facial expressions, as well as with word emphasis and voice inflection. In so doing, that moderator gets a deeper understanding and a clearer view of participants’ attitudes, self-reported behavior, and the emotional basis for their actions, over and above what is afforded by verbal data alone. Surely, this is a characteristic of in-person methodology of qualitative research that cannot be argued. However, considering all that has to be done (and paid for) in order to obtain these subtleties and nuances of “non-verbal data,” the question as to whether it is worth it may be posed.
If we list all the things necessary to collect non-verbal data in a qualitative research study, they will include:
• Requiring all that is involved in the travel aspects of in-person data collection such as the costs of airfare and lodging, rental cars or taxis, meals, getting to one’s hotel, finding the facility, working late into the night, being out of the office, getting through airport security, and potentially dealing with different time zones and associated fatigue
• Paying a focus group facility more than an online panel company to use its database to recruit participants
• Needing two weeks (industry standard) for recruitment rather than one week if an online panel company is used for recruitment purposes
• Imposing scheduling and other logistical challenges for everyone involved including the moderator, his or her client (research manager), internal corporate clients (marketing/advertising executives), and the participants
Indeed, so many different logistical factors all need to be aligned and synchronized in order for this type of study to be successful, and so many additional costs are incurred in the process. For example, take the typical scenario that unfolds in conducting a set of focus groups.
Traditional Focus Groups
The study is approved by the client and then screener development becomes the first step in order to enable the recruitment process. During that two week period of recruitment, the question guide is developed, participants are recruited, clients are notified to make travel arrangements and schedule adjustments, and everyone converges at the facility in whatever city has been designated for the groups to be held. Participants show up (hopefully) and for two hours (industry standard) the entire group is focused on the data collection process. Assuming there are ten participants in the focus group room, each one of them shares their views and opinions for about 10 to 11 minutes (assuming about 10 minutes are spent during the introductory section) to allow everyone in the group to participate.
The (pair of) groups are done for that day and data collection is either complete or, perhaps, everyone is off to a different city for other focus groups to be conducted. At some point, all data will have been collected and the moderator then begins the analysis and prepares the final report. On average, each consumer focus group conducted had an associated cost of approximately $8,000, or $800 per participant, including respondent incentive
In-Person In-Depth Interviews
In comparison, in-depth interviews (IDIs) cost about the same per interview but provide about five to six times the amount of data per respondent, assuming each IDI lasts between 50 and 60 minutes. However, even though there are “more data for the same research buck” to be had with IDIs, they are not used as often as focus group studies partly because they are more time consuming or require multiple moderators to collect the data within the same time period (if we equate four groups of 10 participants to 40 individual IDIs).
All things being equal, IDIs provide more for less considering the balance between cost and amount of data that are yielded. Focus groups are more prevalent because they can be done in a relatively shorter period of time, but do not delve as deeply into participants’ experiences and feelings since the amount of interaction-per-respondent is at least one-fifth that of IDIs. Nevertheless, both can provide non-verbal data.
Online Qualitative Bang for the Buck
Now consider collecting qualitative research data online. Assuming no webcams are used in the process, there are no non-verbal cues that can be obtained using this methodology. The subtleties and other nuance by-products yielded by the in-person method are eliminated from the data collection process. However, for approximately the same cost ($800 per participant, including incentive) each participant can be given the time to provide about 120 minutes of input. Relative to focus groups, this is about 10+ times the amount of data; compared to IDIs, it is at least twice as much.
In this case, the moderator can have a virtual discussion with each person lasting two hours. To facilitate a smooth data collection process and prevent participant drop-off, the required time participants must provide can be spread out over a few days. However, all data can be collected over the same time period (the same week). Also, the number of participants that can be included can be in sets of 10s or even 100s. Given the asynchronous interactions that occur between the moderator and the participant over the course of the study, everyone can fulfill their responsibility during a time that is convenient for them. In essence, each party involved can be either inputting or analyzing data at 3:00AM in their underwear or whatever else suits them.
Online Qualitative Data Quality
Moreover, in many ways, the quality of the data may be enhanced by the online platform and its associated study design specifications, relative to in-person methods. One of the benefits of online qualitative research is in sustaining interaction with participants, which enables us to see longitudinal behaviors, and therefore capture experiences as close as possible to the time that they happen. In reality, the things we are most interested in learning about consumers happen at times when it can be impossible to predict and schedule in advance during the course of everyday life. But with a specially designed “blogspace,” the data can be collected within the context of the phenomenon under study, e.g., in-home usage of a product. So rather than de-contextualizing data collection, as is done by using the artificial setting of a focus group facility, the moderator can be “at the right place at the right time” with those questions that align with the study objectives. In addition, memory decay is eliminated since we can be on the spot at the time the behavior in question is exhibited as opposed to asking participants to put on their memory caps and tell us the story about what happened at some point in the past that is relevant to the study, as is the case with traditional, in-person focus groups and IDIs.
Now consider the effects of the moderator’s presence during data collection, as well as those of fellow participants and clients in the back room behind the big mirror. This condition has the potential to bias data because strong social norms can lead participants to adjust or change their answers so as to appear socially acceptable or politically correct. Online methods help to break down this social desirability bias because the elimination of direct, in-person or telephone interaction provides anonymity which facilitates honesty and truthfulness and allows us to explore even the most sensitive topics.
Surely, online focus groups have been in use for a long time but are only used as a “next best thing to being there” in the case when the type of participant is geographically dispersed. These studies, too, have their own inherent flaws such as scheduling and technological challenges. But nowadays, a confluence of factors has emerged to create the conditions for a new type of online qualitative study, one that leverages the widespread comfort level with blogging and social networking behavior and the penetration of broadband connectivity and digital technology in the homes of general population consumers. In essence, even the most in-depth qualitative research of them all, namely ethnographies, can be done online.
Online Ethnographies
Consider that an ethnographer routinely immerses him or herself in the culture of the behavior under study by traveling to the participant’s home to pose questions, observe the key behavior, collect artifacts that symbolize the subject, prepare field notes, and bring all these different types of qualitative data to bear in the analysis and interpretation for the benefit of the sponsoring organization.
But to optimize the trade-off result between cost, quality, and speed in qualitative research services, a specially designed online platform may be used to achieve the same set of objectives and fulfill on even the same set of deliverables as a traditional ethnography, at a fraction of the cost and time needed. Surely, consumers are able to be recruited, incented, given a set of tasks requiring their text-based responses to questions (blogging), upload photos taken with their cell phones to a website to serve as artifacts, and share their feelings and opinions on a large set of topics. They can do this over a sustained time period and in an anonymous setting. What we leave behind in the process are those highly expensive and, otherwise, intangible qualities that only an in-person data collection method can provide – the non-verbal data.
Bottom-Line
For that matter, what are we really giving up in this “conjoint on qualitative research” that is being addressed here? Indeed, are the non-verbal data so highly valued? Are they so unique and do they provide such a competitive advantage to the organization that collects them? Really, not so much. It is on a rare occasion, at best, that the non-verbals are discussed at length among the analysts and users of all the data. Even less frequently are they mentioned in final reports or become used as part of the strategy or tactics that organizations do as a result of conducting the research. Consider instead, that quantitative research has been leveraging technology for decades, moving from in-person, to phone, to the Internet and driving down costs and study cycle times, while focus groups are still being held the same way as they have over the same decades-long time period. Bottom-line, for the same cost of a few focus groups and in less time, ethnographies can be done yielding much deeper consumer insights, without all the schlepping that has become part and parcel of what we have been doing as market research professionals for so long.
