Background
When recruiting testers from Maze, the quality of test results you receive is our biggest concern. Our goal is to ensure that you get results quickly and that you can trust the results to confidently make informed decisions based on the data you've collected.
Our hired testers do sometimes disappoint and when they do it's important that you let us know so that we can review and take additional action if needed.
You can help us maintain the quality of our tester panels by reporting a hired tester from within your project's test results if they haven't met our standards. This article will walk you through how to report a hired tester and when you should do so.
When should I report a tester?
Reporting a tester should be reserved for situations where it's clear that the tester did not complete the test in good faith because:
- They rushed their responses providing nonsensical answers
- They copied and pasted short phrases like "it is good" to complex questions
- They did not answer questions at all
- They gave up on every mission without seeming to try
In general, you want to give testers the benefit of the doubt. Testers who legitimately struggled due to confusion should not be reported. A high number of misclicks alone is not sufficient reason to report a tester for example.
Distinguishing between a genuine outlier and a reportable tester is important as outliers can still provide insight into the usability of your design while low-quality testers just add noise to your data. These low-quality testers produce data that cannot be used safely.
How do I report a tester?
While reviewing your test results you can report a tester by doing the following:
- Click into the Testers tab to review individual responses
- Click on the hired tester that you would like to report
Note: Hired testers will have the "Paid session" label beneath their tester ID in the list of testers while testers you source or hired testers who aren't paid for their test will simply have "Test session" for a label - While viewing a hired tester's response, click the triple dot menu in the upper right
- Click Report tester as shown below:
- Provide the requested information as shown in the form below:
- Click Report to submit your report
What happens after I report a tester?
Your report will automatically alert our Support team and we will begin investigating your report by reviewing the responses provided by the tester. We ask that you please leave the tester's response intact and undeleted for at least a business day after submission to give us time to review their responses.
If we confirm the validity of your feedback, we will then review the tester's history to see if they've been reported before and whether there's a pattern of low quality testing. Depending on what we find, actions taken range from issuing you a credit for the reported tester to blocking the tester from future panels.
Once we've made a determination, we will reach out to you with our findings.
Why can't I report a particular tester?
Hired testers only get compensated if they complete your test from beginning to end. If a hired tester abandons the test before completion, you are not charged for the test and we'll continue sourcing testers until we complete your order of tests. For this reason, testers who do not complete the test are not classified as "Paid sessions" when viewing the list of testers and cannot be reported.
Instead, you can simply delete these test sessions.