Mistakes in qualitative research, like biased questions, incomplete data, and ignoring context in IDIs, FGDs, and Diary Studies, lead to flawed conclusions and unreliable outcomes. Addressing these ensures findings are accurate, reliable, and reflective of genuine user experiences.
Conducting user research is essential for understanding user needs and behaviors, but mistakes in data collection can skew findings and lead to poor design decisions. Whether you’re using methods like In-Depth Interviews (IDIs), Focus Group Discussions (FGDs), or Diary Studies, it’s crucial to avoid common data collection errors. Here are the most frequent mistakes and how to steer clear of them.
One of the most common mistakes is asking leading questions that guide participants toward specific answers. For example, asking, "Isn’t this feature useful?" implies that the feature should be useful, which may influence participants to agree, even if they don’t genuinely feel that way.
How to avoid it: Stick to neutral, open-ended questions that let participants express their opinions without being steered in a particular direction. Instead of, "Is this feature helpful?" ask, "What are your thoughts on this feature?" or "How would you describe your experience using it?"
Related: What should you avoid when framing research questions?
Another mistake is not digging deeper into participants' responses. When a participant gives a brief or surface-level answer, researchers sometimes move on without understanding the full context. For example, if a participant says, "I don’t like this feature," it’s important to ask why, rather than accepting it at face value.
How to avoid it: Use follow-up questions to probe deeper. Ask for clarification or examples. For instance, if someone says, "This app is confusing," ask, "Can you describe a specific instance where it was confusing?" This helps uncover valuable insights.
Providing too much information at once can overwhelm participants and result in rushed or shallow responses. This often happens when researchers present complex scenarios or lengthy product descriptions without giving participants time to process.
How to avoid it: Break information into smaller, digestible chunks. Allow participants time to absorb and reflect before responding. Check for understanding along the way to ensure they are comfortable with the discussion.
In IDIs or FGDs, it’s easy to miss important details or misinterpret responses, especially if the researcher is taking notes manually or trying to capture everything on the spot. This can lead to incomplete or inaccurate data.
How to avoid it: Always record sessions with participant consent so you can revisit the data later. Take detailed notes during the session, but don’t rely on them alone. Recordings allow for a more thorough analysis afterward.
Related: Our favourite research analysis platforms
Diary Studies require participants to log their activities and thoughts over time, but relying too heavily on participants’ memory can introduce recall bias. For example, participants may forget to log details immediately and later struggle to remember accurately.
How to Avoid It: Encourage participants to record their entries in real-time or as soon as possible after an event. Provide simple, user-friendly tools like mobile apps or prompts to make logging easier and more frequent.
Related: What are best practices when using the diary method?
Research conducted in artificial or controlled settings can miss the impact of context on user behavior. For example, users may behave differently when using a product at home versus at work, but these differences can go unnoticed if the research setting doesn’t mimic real-life use.
How to Avoid It: Consider the context in which users engage with your product. In IDIs or FGDs, ask participants about where, when, and how they use the product. In Diary Studies, encourage them to document the circumstances surrounding their actions.
FGDs can suffer from groupthink, where participants conform to the opinions of others rather than sharing their true thoughts. This happens when a dominant participant sets the tone, and others feel pressure to agree.
How to Avoid It: Create a balanced discussion by encouraging all participants to contribute. Make space for quieter voices and ask individuals to write down their thoughts before sharing them with the group to avoid early influence from others.
Related: What are the pros and cons of focus group discussions?
Verbal responses are important, but non-verbal cues—like facial expressions, body language, and tone of voice—can provide deeper insights into how participants feel. Ignoring these signals can result in missed opportunities to understand users' true emotions.
How to Avoid It: Pay attention to non-verbal cues during interviews or FGDs. Take note of hesitations, changes in body language, or facial expressions, as these can offer additional layers of meaning. Recording video (with consent) can help you review these cues later.
Avoiding these common mistakes—leading questions, incomplete data, recall bias, and groupthink—ensures more accurate and reliable user research findings. By carefully planning your approach, probing deeply, and paying attention to both verbal and non-verbal cues, you can collect richer, more actionable insights that truly reflect user needs.
Cover image by George Milton
