Research: tree testing design system site IA

[martinwake]

Background

In January 2022 we used UXTweak to set up a tree test for a proposed reorganisation of the design system website (#329, Site IA: Review and update).

The proposed reorganisation would

• Merge the Backlog, Contribute and Roadmap sections into one section called "Community"
• Add a "Get started" section

The main sections would then be:

- Get started
- Components		
- Patterns		
- Community		

The test had 10 respondents (8 content designers, 2 interaction designers). They completed 3 tasks presented in a random order and were also given the opportunity to add any other comments they might have on the site or its organisation. All questions were optional.

The tasks were:

1. You want to find out what new work is planned for the design system. Where can you find this information?
2. You want to suggest a new pattern to be included in the design system. Find out how to do this.
3. You've found a pattern that you want to use in your prototype. Where would you find out how to do this?

(It was probably a mistake to include the word "pattern" in tasks 2 and 3, when there is a section called "Patterns". This can affect results by cueing people to start at that section. The effect is probably that failure rates for tasks 2 and 3 will be slightly higher than they otherwise would have been.)

Detailed results for each task

Task 1: You want to find out what new work is planned for the design system. Where can you find this information?

Correct answer: Community > Roadmap

8/10 respondents correctly chose the Community section first; the other 2 chose "Get started".

Within the community section there was confusion with the Backlog and Roadmap items: 2 respondents said they would look in Backlog for this information while one indirect success also considered Backlog as an option.

Task 2: You want to suggest a new pattern to be included in the design system. Find out how to do this.

Correct answer: Community > Contributing

5/10 people went directly to the correct section. However 3/10 people went directly to Patterns > Add another. Should we rename this pattern?

The remaining 2 people both went to Patterns first and then backtracked to find the correct section. One also tried Get started before trying Community.

We should make it possible to add a new pattern from the pattern page itself.

Task 3: You've found a pattern that you want to use in your prototype. Where would you find out how to do this?

Correct answer: Get started > Copy HTML and CSS into a prototype

Less than half the participants found the right section, with 9/10 starting in the Patterns section and 6/10 nominating that as their answer.

3/10 chose Patterns > Add another as the way to do this.

No two participants chose the same path: there is a lot of confusion with this task.

Other comments

Participants were asked a final question:
Is there anything else you'd like to tell us? This could be a comment on the site structure, something you were expecting to find but couldn't, or any other thoughts on the organisation of the Design System website.

Respondent 4

I think I'd expect to be able to add a pattern in the patterns section as well as the community section. I think I made the wrong choice for the use a pattern question - I think I'd expect to be able to use it from the patterns section, under the specific pattern I was looking for, but something made me shy away from that because the question didn't mention a specific pattern!

Respondent 5

I was confused about the second task, (adding a pattern into my prototype). If expect to be able to go to the pattern first and find out how to add it to my prototype from there.

Respondent 6

The information at the start of each page, for example on components. The main reason i'm on that page is to find the component i'm looking for, I want to view that table. That initial wordy information at the top is only useful to me for the first time I visit, and it's wordy, I fly past it looking for what I want. Is there a better way for that information to be highlighted/presented outside of taking up the main 2/3rds of my vertical screen estate in the main body, when 98% of the time it's not my main purpose on the page.

Respondent 7

For the second task I would have expected a link on the page where I found the pattern to the. 'how to use this in your prototype' page.

Respondent 8

In the navigation, I usually expect the 'contribute' link to be last (just because it's like that in the GDS one, so am conditioned to look for it there!). It may be good to have the roadmap in between the latest updates and the backlog in the navigation. I like the purple, makes it easy to spot that I'm in the DWP design system as I have links to that and the GOV.UK one next to each other in my bookmarks :)

Respondent 11

I'd love more examples from a variety of real services.

Summary and actions

Task 3 scored lowest. This may be a feature of how the task was worded; starting them off with having found a pattern, perhaps people were thinking that they'd be in the Patterns section already.

If the task was "You're using the design system for the first time" then perhaps they would have succeeded more.

However - one clear finding is that some people expect to find the pattern first and then work out how to use it, plus being able to access guidance from the pattern page itself. We could add a standard "How to use this in your prototype" link to all components and patterns.

Answers to our initial questions

Where should the "Team" page go?
Not tested.

Where should the "Updates" articles go?
8 out of 10 people correctly chose "Community" as the section to look in. However within that section they looked in both "Backlog" and "Roadmap".

Other solutions:

• Home Office design system doesn't publish a roadmap.
• GOV.UK puts it in Community.
• NHS has a "What's new" section with Updates and Blog posts (aggregated from other platforms); the Backlog is in Community and Contribution
• IBM has What's happening > Releases, a Backlog on Github and a Roadmap on Zenhub - both links, not reproduced on the site. They also have a Release Radar on GitHub.com: https://github.com/carbon-design-system/carbon/wiki/Release-radar - using the Wiki on GitHub

Where should the "Get started" pages go?
Despite the fact that people went the wrong way for this I think we can keep the same architecture - the task wording may have steered them towards Patterns as the starting point. However we should add links to the pages from each Pattern page.

Revised proposed structure

Get started
	- For designers
	- For developers
	
Components (alphabetical list)
	- Component 1
	- Component 2
	---
	- Component 3 (retired)
	
Patterns (alphabetical list)
	- Pattern 1
	- Pattern 2
	---
	- Pattern 3 (retired)

Community
	- Roadmap (current)
		○ [previous date]
	- Backlog
	- Contribute
	- The team

[jonhurrell]

We undertook and completed the structure changes based on the insights here.