Test for understanding

Usability testing, a core part of user experience (UX) research and design, is a helpful technique when people have to find your information in order to understand it. With usability testing, you test the content as a whole, not just individual paragraphs.

In this guide, you will find a brief overview of usability testing. For more detailed information on usability testing and requirements and procedures for conducting testing, please view Digital.gov’s usability resources and Supporting Usability Testing Through Paperwork Reduction Act Flexibilities (PDF, 110KB, 6 pages). Talk to your agency experts for any agency-specific UX processes.

When to test

You can conduct usability testing at any time that you have draft content. After you make changes based on the first round of usability testing, you can conduct a second round to see if your changes solved the problems you found without introducing new problems.

Who to test with

You need to find three to five people to test your website or document.

Identify who your intended readers are. For example, individuals searching for medical information; taxpayers and tax professionals looking for forms; travelers wanting a passport.

Develop simple criteria and find three to five people who match them. For example, for travelers, the criteria might be: Adult U.S. citizens who haven’t applied for or renewed a passport lately. The criteria don’t need to be complicated.

Using your network of colleagues, friends, and family, find people who, more or less, meet the criteria and will give you an hour of their time. Don’t use members of your own team, but employees from a different team down the hall may be fine.

You're not required to get any special permission to do a usability test with a small number of people. Set aside a morning to conduct your test, and give each of the volunteers an appointment, one hour apart from each other.

Session overview

A typical usability test session lasts about one hour with these three parts:

Introduction

You make the participant comfortable, explain what will happen, and ask a few questions about the person to understand their relevant experience.

Scenarios

You give the participant very short stories suggesting they have a need for specific information and then you watch and listen as they find that information and tell you what they understand from what they found. An example of a scenario for the FAA web site might be:

You have a private pilot’s license and you just moved to a new city. Find out if you need to tell the FAA about your new address. If you do, find out how to do that.

You can also ask participants for their own scenarios. What would they come to the site you are testing to find out? Then watch and listen as they look for and try to understand the information.

Typically, you ask people to “think aloud” as they work so you hear their words for what they are looking for and you hear how they understand what they find.

Debriefing

At the end, you can ask neutral questions about the experience and follow up about any specific words or phrases.

Testing variations

Variations on the one-on-one usability test:

• Two people working together (co-discovery). Their discussion is an easy form of thinking out loud.
• Several people working independently at the same time, followed by a group discussion. This speeds up the time you spend in usability test sessions, but it only works if you have several usability test note-takers so you have someone watching and listening to each participant before you bring all the participants together for the discussion.
• Comparative usability tests. You can include different versions of your document. Because you have a small number of people, it is best to have each person work with both versions. You have to alternate which version people start with. (See Controlled Comparative Studies below)
• Remote moderated usability testing. With web-based tools, you do not have to be in the same place as the participant. These tools allow you to draw participants from a wide geographic range without travel costs.
• Remote unmoderated usability testing. You can have large numbers of people participate through remote testing tools. (For federal agencies, depending on what you are testing, this may require clearance through OMB.)

Usability testing from the National Cancer Institute

The information was good, but the title confused people.

A team at the National Cancer Institute tested a brochure on skin cancer prevention. They wanted to make sure that the information, title, and design images worked together well. One of the important messages was that even people with dark skin can get skin cancer. People understood the information in the brochure, but said that the title, “People of Color Get Skin Cancer, Too,” made them think it was only for African-Americans.

The team changed the title to “Anyone Can Get Skin Cancer” (along with some other changes from the usability test recommendations) and tested it again. This time, when people were asked who the information in the brochure was for, they correctly identified many different people. More importantly, they all said that the information was “for them,” too.

The team included both plain language experts and medical subject matter experts. This case study illustrates three points:

• Plain language experts test their work.
• Even a small change can make a big difference in the success of the project.
• Retesting after you make a change is important. The second test may validate your decisions; it may also suggest additional changes.