Get the most out of Tree Test questions by following these tips.
- Tree Test questions are available in modern surveys only.
- Tree testing is available at an additional cost. If you are interested in this feature, please contact your Customer Success Manager.
Test early and test often
- Before redesign work.
- After a Card Sort activity to validate the information architecture results.
- Before generating content or layouts.
- At all stages of the design process whether you're testing a prototype, building something new, improving an existing structure, or checking that your information architecture is still working as intended.
- To refresh stale information architecture that might not be meeting your needs anymore. For example, if your company has added new products or deprecated old ones, you may want to test an updated navigation tree that reflects those changes.
Use your best judgement on what a good-sized tree should be
Technically, there is no tree that's too big or small to test. Practically speaking though, very large trees or nodes that go ten levels deep are not ideal. Such Tree Test questions are challenging to navigate on desktop or mobile. And if these structures are challenging to test and collect data on, imagine how difficult navigation would be in a real prototype or app. If you find yourself authoring a very large Tree Test question, consider whether you should simplify the tree.
Write scenario- or task-based questions that don't give the answer away
Describe a scenario or a task that participants must perform while interacting with the tree. For example, "Imagine you are checking our online grocery shopping website to see if there is a special promotion on your favorite soda. Where would you expect to find the information?"
Avoid using words in the question text that are perfect matches for the labels in the tree. Doing so will prime participants to scan the tree for keyword matches instead of thinking more deeply about the task. In the example above, the information might be under Sales, Weekly Flyer, or Drinks in the tree, but none of these words appear in the question text.
Test one tree per activity
- The nodes, child nodes, or levels are structured differently.
- The menu and item labels are different.
If a participant sees Tree A while completing the first task and sees Tree B while completing the second task, having seen Tree A may unintentionally influence them as they evaluate Tree B.
- Create separate surveys
for different trees.
This approach is ideal if you have multiple tasks you want participants to complete.
- Use
Least Filled concept
testing.
This approach is ideal if you only have one task. Each tree can be its own concept within a single survey.
Collect information on users' state of mind
Tree test reporting contains valuable usability information such as success and failure rates and participants' paths to success. However, it gives no insight into what participants are thinking as they complete the task. Are they confident in their first click, or just making a best guess? Were they evaluating multiple options, and if so, what were they?
-
Add a Recording
action to your survey preceding the Tree Test question.
Ask participants to think out loud as they complete the task. You'll have recordings and transcripts to review later.
- Ask open-ended follow-up
questions in your survey.
Example
What are your thoughts on the labels of the top-level nodes?
What frustrated you most about this task?
If you had a magic wand, how would you improve the navigation?
What did you like about the navigation?
What came to mind when you had to [task]?
What would enable you to accomplish [task] more effectively?
What are your overall impressions of the navigation?
Do you have any final thoughts on the task or on today's activity?
- Ask closed follow-up
questions to capture participants' state of mind in a structured way.
Tip: If you're expecting a small sample size, closed follow-up questions may be a better approach than open-ended follow-up questions. This way, you get actionable quantitative data points.
Example
After a task, ask a Single Choice question:
Which of the following statements best describes your decision-making? - There is only one possible place I thought I could find this information.
- I wasn't sure where I would find it and I just made my best guess.
- There was more than one option that I thought could contain the information.
If the participant selects the second or third answer option, ask this follow-up Multiple Choice question:
You mentioned you were deciding between more than one or weren't sure how to best find the information. Which were you considering? - List the possible nodes they could have selected.
Optimize the Tree Test user experience
- Exclude generic labels
like Contact Us, Help, and Search.
Although these options are useful in real life, in a Tree Test question, participants might select these as an easy way out of finding the answer.
- Aim to have at least 50
participants.
The number of people for tree testing depends on the size of your tree, the diversity of your user base, and the resources available to you. If your user base is very diverse, or if your tree is large or complex, you might benefit from having more participants. Remember, the quality of your participants is just as important as the quantity.
- Only show participants one tree per activity to avoid creating unintentional bias.
- In an activity with
multiple tasks, limit the number of tasks to 5-8.
Be mindful of the overall cognitive load you're asking of participants. If the trees are large or complex, keep the number of tasks on the lower side.