When recruiting panel participants, the quality of test results you receive is our biggest concern. Our goal is to ensure that you get results quickly, and that you can trust the results to make informed decisions confidently based on the data you’ve collected.
Maze uses third-party providers for its panel features. We partner with Prolific and Cint for unmoderated panel testing, and Respondent for moderated panel testing (currently in beta testing).
If you’re not happy with the quality of the responses submitted by a panel tester, we encourage you to report it. This helps us maintain our panel quality.
In this article:
- When should I report a participant?
- What happens after I report a participant?
- Why can’t I report a particular participant?
When should I report a participant?
As a general rule, try to give participants the benefit of the doubt. Those who legitimately struggled because of confusion shouldn’t be reported. For example, a high number of misclicks isn’t a sufficient reason for reporting.
Distinguishing between low-quality responses and a genuine outlier is important. Outliers can still provide insight into the usability of your design. Participants responding in bad faith just add noise to your results and produce data you can’t use safely.
You should report when it’s clear that the participant didn’t complete the test in good faith. Some examples:
- The replies were submitted by a bot
- Clips were required, but they haven’t allowed recordings, or the quality was insufficient
- They didn’t follow task instructions
- The participant profile didn’t match the specified filters
- The device used to take the test was different than the one specified
- They they haven’t answered in good faith, providing copied, rushed, or nonsensical answers, or repeated short generic phrases like “Good” to complex questions
- They gave up on every mission without seeming to try
How do I report a participant?
To report a hired tester:
- Open your Results dashboard.
- Open the Participants tab to review individual responses.
- Select the specific participant that you’d like to report.
Participants who completed the maze appear in the results as a ‘Panel session’. However, those who started the maze but haven’t completed it appear as an ‘Anonymous session’ instead. Reporting is only available for panel sessions. Why can’t I report a particular tester?
- Click Report this tester.
- Select the reason for reporting, and add any relevant details.
- Click Report to submit your report.
What happens after I report a participant?
Prolific
If you report a participant sourced from Prolific, the Maze Support team will be notified automatically. They’ll investigate by reviewing their responses and history.
After submitting a report, please keep the affected response intact and undeleted, so we have time to review it.
Depending on our findings, we may take actions ranging from issuing you a credit refund for the reported participant to blocking them from future panels. Once we’ve made a decision, we’ll contact you with our findings.
Please note, you’ll need to manually place a new order to replace any refunded participants.
Cint
If you report a participant sourced from Cint, you’ll be automatically refunded, and a new order will be put in place.
Why can’t I report a particular participant?
Panel testers only get compensated if they complete your test from beginning to end. If they abandon the test before completion, you aren’t charged for the test and we’ll continue sourcing testers until we complete your order.
For this reason, testers who haven’t completed the test aren’t classified as a ‘Panel session’, and therefore it’s not possible to report them.
You can simply delete those test sessions if they aren’t relevant to your results.
Still need help?
If you have any questions or concerns, please let our Support team know — we'll be happy to help!