There are two main approaches to user testing: moderated and unmoderated.
Both methods have benefits and disadvantages. Choosing the right approach depends on various factors, and isn’t always a clear-cut decision.
This article briefly explores some guidelines to help you decide whether to opt for moderated or unmoderated testing.
Unmoderated testing
In an unmoderated test, participants go through the test independently, without the direct guidance of a moderator. Unmoderated testing can be more suitable in these scenarios:
- Larger audience sizes: If you want to test with many participants, unmoderated testing is more easily scalable to a higher volume of participants.
- Tight deadlines: When time is of the essence, unmoderated testing allows you to collect feedback and validate ideas efficiently, without scheduling constraints.
- Budget: Moderated testing tends to be costlier and more time-consuming to implement. If your research project has a lower budget, unmoderated testing may be a better fit.
- Shorter tests: If your test can be completed in under 15 minutes, you can expect participant engagement to remain fairly high. In these cases, unmoderated testing can be a convenient option that is less resource-intensive for your team.
- Higher-fidelity prototypes: You can test high-fidelity prototypes in both moderated and unmoderated settings. However, during later design stages, unmoderated tests offer several advantages. They enable you to measure usability metrics consistently, and allow you to observe testers' natural behavior without introducing bias. Users typically interact with products without a moderator: running unmoderated tests with high-fidelity prototypes can better simulate real-life experiences. In moderated sessions, participants are more conscious of being observed, potentially affecting their behavior or encouraging them to seek help. Moderators might also find it challenging to navigate uncomfortable silences or guide the conversation without inadvertently influencing participants.
Moderated testing
Moderated testing involves having a researcher or design expert guide participants through the test. Here are some situations where moderated testing can be beneficial:
- Limited topic knowledge: In early design stages, when you don't know much about the topic, moderated testing helps facilitate open-ended, exploratory discussions with participants.
- Less certainty about expected behavior: If you're unsure how users will react to your design, a moderator can observe closely and ask clarifying questions as users interact with it.
- Unfamiliarity with tech: If your target audience is not so comfortable or familiar with technology, a moderator can support them in navigating the test.
- Longer tests: In unmoderated tests, attention drops as tests get longer. For lengthier tests, moderated sessions help maintain participant engagement.
- Strategic topics: For complex or strategic topics, consider a mix of both approaches. Run a moderated session to gain in-depth insights, followed by unmoderated testing to gather a larger sample size.
Which approach is best?
In nearly every situation, there’s no approach that's inherently better than the other. It all comes down to what fits your team’s processes and capabilities.
Ultimately, teams who do research can establish their own guidelines based on their specific team characteristics and project requirements. It’s helpful to align on and document criteria to help your team decide which approach to take.
A few examples of criteria are:
- Budget: Consider the monetary and human resources available for conducting the research, and the associated costs of each approach.
- Deadline: Consider the time constraints and urgency of the project. Unmoderated testing might provide faster results if you’re on a tighter timeline. Moderated testing could offer deeper insights at the cost of a larger time investment.
- Project stage: Early stages may benefit from exploratory qualitative methods, while later stages might demand quantitative insights.
- Audience: Assess the characteristics, preferences, and familiarity of the target audience with technology and your product.
- Importance of the decision being made: Reflect on the significance of the insights you need to gather and the impact of the decisions relying on the research outcomes.
- Knowledge gaps to be filled: Get clarity on the information gaps that the research should address. Validating a design decision may be addressed effectively with unmoderated research, whereas developing an understanding of an unknown market may not.
Developing a method that combines both moderated and unmoderated approaches could be a highly effective strategy.
For instance, you can intercalate qualitative and quantitative methods in a sequence: moderated interview-style questions, followed by unmoderated usability testing, concluding with another round of moderated conversations.
In the early stages of a research project, engage participants in exploratory conversations that allow you to uncover insights about user behaviors, motivations, and pain points.
You can then move on to focus on collecting quantitative usability metrics and identifying broader trends in user preferences, time spent on tasks, and success rates.
Close the loop with another round of moderated testing to deep-dive into the "why" behind the quantitative findings. By interviewing participants, researchers can encourage participants to think aloud to explore the reasoning behind certain behaviors, dive into unanticipated insights, and gain clarity on any contradictory quantitative data.
After launch, you can continue to leverage both approaches to gather insights on your live products and continue to iterate on it.
Additional resources
- Read more about this topic in our Beginner's Guide to Usability Testing
- Learn how to build a strong foundation for effective research with the Research Maturity course