Reporting on Card Sort question results – Alida Help Center

Learn more about viewing Card Sort data.

Access reporting

You can view your results in Monitor and modern reporting. Monitor displays up-to-date participation data as responses are coming in. Modern reporting allows you to analyze Card Sort responses in greater detail.

To view participation data, click Monitor.
Card Sort questions are listed as part of the survey path. Completion, skip, and stop point information is displayed for each of them.
To view response data, click Report.
Under Modern Reporting, click Get Started.
Create a new modern report or open an existing one.
Scroll to the Card Sort question tile.

Card Sort question tile

The Card Sort question tile displays a similarity matrix that shows how often cards were grouped together.

Think of the card names as labels for columns and rows.
Think of the similarity matrix as a consensus map. Each cell represents the strength of the relationship between two cards.
Focus on one cell in the similarity matrix at a time. With your focus on the cell, scan for the card names at the top of the column and to the right of the row. The percentage value in the cell represents how often these two cards were grouped together. For example, in the screenshot below, 74% of responses paired the cards Lip Oil and Lip Liner in the same category.
Blue cells indicate cards that were frequently paired together. The darker the shade of blue, the stronger the association between the two cards.
The blue shading also means you can read the similarity matrix like a heatmap. Focus on the areas of the similarity matrix where all the dark blue cells are clustered together to identify which cards are more likely to be grouped together.
Hover over the cell to view a tooltip with more detailed insights.
Click the cell to view the full list of categories in which both cards appear.
Click Open in Full Screen to view the full table.
Filter results to uncover specific insights.
Export the results to CSV for further analysis.

Tip: As you're evaluating your Card Sort data, use these questions to guide your analysis:

Does the data validate your hypotheses about information placement and architecture? If not, what do you need to change?
Are there unclear or misleading category labels that seem to confuse participants? What can you do to fix them?
Which categories were most ignored? Is there an opportunity to simplify your information architecture by eliminating them?
Which categories were overused? Is there an opportunity to split those into further categories?

Participant-Centric Analysis (PCA)

Click Participant-Centric Analysis to view the most popular mental models or information architectures (IAs) arising from participants' Card Sort results.

The Participant-Centric Analysis tab displays the top five most frequently occurring IAs.

Each IA box contains a Submitted by Participant response. This mental model is considered the most representative of the responses grouped together under that IA.
The fraction at the top of the IA box represents the number of participants whose Card Sort results were similar to this participant's. This fraction excludes the Submitted by Participant response. Therefore, you can think of the real total for each IA box as the numerator plus one.
The number of responses represented by all the IA boxes may not add up to the total number of responses. Responses that fall outside of the top five IAs are excluded.
Within each IA box, you can see the number of categories, and the number of cards in each category.
Expand a category to see similar category labels and which cards were sorted together.
Alternately, click the expand all categories and collapse all categories buttons in each IA box.
Click Open in Full Screen to view all five IAs in a larger screen.

How are the IAs calculated?

To determine the top five IAs displayed on the Participant-Centric Analysis tab, the application uses a k-means clustering algorithm.

The algorithm looks at the card pairings in each participant's response and converts them into a binary vector. A value of 1 is assigned to a card pairing. A value of 0 is assigned to a non-pairing.
Example

4 cards (A, B, C, D) = 6 possible pairs
```
Participant who grouped {A,B} and {C,D}:
  Vector: [AB=1, AC=0, AD=0, BC=0, BD=0, CD=1]
Participant who grouped {A,B,C} and {D}:
  Vector: [AB=1, AC=1, AD=0, BC=1, BD=0, CD=0]
```
The algorithm groups participants with similar binary vectors into clusters.
If two participants are in the same cluster, it means that they think about the cards in a similar way and often make similar mental associations. Their mental models support each other.
Within each cluster, there is one participant that's closest to the center of the cluster. This participant's response is the Submitted by Participant mental model displayed in the IA box.
The algorithm counts how many responses are in each cluster.
Whether a response belongs to a cluster isn't determined by a hard-and-fast rule. Instead, the k-means clustering algorithm looks at relative similarity (for example, This response is more similar to the center value in cluster A than the center value in cluster B or cluster C). The boundary for each cluster will adapt and be slightly different, so that the responses that naturally cluster together are counted under one IA.
Note:
- The threshold for similarity is set to a very conservative value of 0.0001. This means that variance within a cluster is minimal. Similar binary vectors have to be very similar to be counted as one cluster, so you can have a lot of confidence in the IA groupings.
- In cases where the number of responses is too low, the algorithm may have difficulty distinguishing clusters and end up grouping dissimilar responses together.
The algorithm identifies the top five IAs with the most support and the center values for each of them. These are represented by the five IA boxes.

Access reporting

Card Sort question tile

Participant-Centric Analysis (PCA)

Example

Related articles