Whether to run unmoderated studies or in-product surveys in your next user research depends on multiple factors including learning goals, the type of data you want, stage of the product life cycle, the context of your study, and more. Also, consider using a combination of both methods to have a more well-rounded study as the right method doesn't have to be one over the other.
While there is no straightforward formula for when to use one method over the other, below is a quick reference on the pros and cons of each approach:
In-product surveys
Pros / good for:
- Running summative studies, where you already have a product setup and need to evaluate how it’s doing.
- Getting contextual feedback as you can trigger surveys at specific flow or events happening in the product.
- Targeting specific audience and actual users of the product as surveys can be triggered to specific audience or events happening in the product.
- Collecting quantitative data for statistical significance for guiding your design decisions.
- Additionally recruiting users to participate in follow-up or unmoderated studies.
Cons / Not good for:
- Detailed, lengthy studies because surveys are shown as small modals or pop-ups on the side of the screen, making it difficult for testers to concentrate for too long.
- Early prototype testing as it is difficult to present the prototype. Unmoderated studies are suited for this.
Unmoderated studies
Pros / good for:
- Iterative prototype testing as you can insert Figma prototype links and tasks.
- Tasked-based tests where you want to observe how users go through about each task. The study result page highlights the different paths that users take to succeed or fail the tasks. To learn more about making sense of prototype task results in an unmoderated study.
- A/B testing to gain feedback on design variations. Learn more about how to run A/B or split testing.
- Card sorting to understand how users mental process and help with information architecture. Learn more about running card sorting test.
- Collecting in-depth qualitative data. Unlike in-product surveys, the full screen mode in unmoderated studies lets more room for participants to easily share feedback.
Cons / Not good for:
- Not as effective as the in-product survey for collecting contextual data.
To learn more, see related articles:
- How to create an in-product survey using Hubble
- How to create unmoderated prototype tests using Hubble