When recruiting panel participants, the quality of test results you receive is our biggest concern. Our goal is to ensure that you get results quickly, and that you can trust the results to make informed decisions confidently based on the data you’ve collected.
If you’re not happy with the quality of the responses submitted by a panel participant, we encourage you to report it. This helps us maintain the quality of the panel.
In this article:
- When should I report a participant?
- What happens after I report a participant?
- Why can’t I report a particular participant?
When should I report a participant?
As a general rule, try to give participants the benefit of the doubt. Those who legitimately struggled because of confusion shouldn’t be reported. For example, a high number of misclicks isn’t a sufficient reason for reporting.
Distinguishing between low-quality responses and real outliers is important. Outliers can still provide insight into the usability of your design. Participants responding in bad faith just add noise to your results and produce data you can’t use safely.
You should report when it’s clear that the participant didn’t complete the test in good faith. Some examples:
- The replies were submitted by a bot.
- Clips were required, but the participant hasn’t allowed recordings, or the quality was insufficient.
- They didn’t follow task instructions.
- The participant profile didn’t match the specified filters.
- The device used to take the test was different than the one specified.
- They they haven’t answered in good faith, providing copied, rushed, or nonsensical answers, or repeated short generic phrases like “Good” to complex questions.
- They gave up on every mission without seeming to try.
How do I report a participant?
To report a hired participant:
- Open your Results dashboard.
- Open the Participants tab to review individual responses.
-
Select the specific participant that you’d like to report.
Participants who completed the maze will appear in the results with a label containing the panel source. However, those who started the maze but haven’t completed it appear as a ‘Dropped off’ session instead. Reporting is only available for those with a panel source session label. Why can’t I report a particular participant? -
Click Report participant.
- Select the reason for reporting, and add any relevant details.
- Click Report to submit your report.
What happens after I report a participant?
Unmoderated
For unmoderated studies, what happens after you report a participant depends on when you file the report:
- Within 72 hours: Once reported, we will automatically look for a replacement participant at no additional charge to ensure your study stays on track. Additionally, any data from the reported participant will be removed from your participant list and results data.
- Outside 72 hours: Once reported, we will refund the credits for the participant back to your account and remove any data associated with the reported participant from your account. However, if you want to replace the participant, you will need to place a new, separate panel order.
Why can’t I report a particular participant?
Incomplete tests
Panel participants only get compensated if they complete your test from beginning to end. If they abandon the test before completion, you aren’t charged for the test and we’ll continue sourcing participants until we complete your order.
For this reason, participants who haven’t completed the test aren’t classified as a ‘Dropped off’ session, and therefore it’s not possible to report them.
You can simply delete those test sessions if they aren’t relevant to your results.
Still need help?
If you have any questions or concerns, please let our Support team know — we'll be happy to help!