The agreement rate is the percentage of users who placed a card in a certain category in a card sort.
Single card agreement rate
The single card agreement rate is the percentage of users who decided that a specific card belongs in a category. An agreement rate of 100% means that all testers agreed that a card belongs in a specific category.
How is the agreement rate calculated?
The agreement rate is calculated using the following equation:
For instance, consider the example screenshot below:
- The card "Key lime pie" was placed in the category by all 8 of the testers. The agreement rate for that card in that category is 100% (8 * 100 / 8)
- The card "Salted caramel ice cream" was placed in the category by 7 out of 8 testers. The agreement rate for that card in that category is 87.5% (7 * 100 / 8)
- The card "Pastel de nata" was placed in the category by 3 out of 8 testers. The agreement rate for that card in that category is 37.5% (3 * 100 / 8)
Average agreement rate
The average agreement is calculated as a simple average between the agreement rates of all cards inside a category.
In the example screenshot, the average agreement for the "Desserts" category is 75%, as that is the average of the agreement rates of all cards inside that category.
The agreement matrix groups cards under each category based on their highest agreement rates. It gives you a visual representation of how often cards appeared in each category. Learn more
The similarity matrix evaluates the relationship between pairs of cards. It gives you insight into which cards were most frequently grouped together. Learn more
How many testers do I need?
In quantitative studies, the more users you can test with, the better the chance your results are accurate. As a rule of thumb, we recommend testing with at least 20 users.
Having more testers can be particularly helpful when doing card sorting: as more testers participate, agreement rates become easier to interpret, and you start seeing more meaningful patterns in the data.
Learn more in this article: How many testers do I need?
Card sorting results
To learn more about your card sorting results, check out this article: Understanding your card sorting results