How to conduct a usability test

This resource provides a script and example questions grouped by the six key moments usually found in a usability test: introductions, warm-up, set-up, task completion, follow-up, and wrap-up. If your participant's time is scarce, consider sharing a few questions ahead of time.

Introduction

Hello, my name’s [moderator], and I am going to walk you through today’s session. As I mentioned over email, my office is currently working with [partner] to [broad purpose; for example, inform a redesign of agency.gov]. Joining us on the call are [observers, and their affiliation].

I would like to begin by thanking you for making time to speak with us. Your feedback is valuable, and will help us determine if our website functions as intended. Just to confirm, we would like to keep this session to [duration; for example, 30 minutes]. Does that still work for you?

If you need a break or to stop at any time, please let me know.

During this session, I will start by asking you a few questions around your work. Later on, I will ask you to share your screen and accomplish a few tasks using the website we are evaluating.

Please be aware that there are no wrong answers. As you go about using the website, I will ask you to think aloud as much as possible: to describe what you are looking at and what you are trying to do. This will be a big help.

Also, please do not worry that you are going to hurt our feelings. We are doing this to improve the site, so we need to hear your honest reactions.

If you have any questions as we go along, just ask. I may not be able to answer them right away, since we are interested in how people do when they do not have someone sitting next to them to help. But if you still have any questions when we are done I will try to answer them then.

With your permission, I would like to record this call. The recording will only be used to help us figure out how to improve the site, and it will not be seen by anyone except team members.

Recording this call also helps me because I can really focus on our conversation instead of being distracted by taking notes! Is that okay?

Finally, I want to confirm that you have received a participant agreement. Have you received the agreement?? Have you had a chance to review it? Did you have any questions? If you have not already, please sign it and send it back at your earliest convenience.

Do you have any questions for me at this time?

Warm-up

To start, I would like to hear a little bit about you:

What is your occupation?
What do you do all day?
What kind of person does your kind of work, generally?
Is this something that requires a specific kind of training or experience?
What are your goals in doing your work?
What motivates you?

Set-up

We are done with the questions, and we can start looking at things. I would like you to share your screen, but before you do, make sure to close anything you do not want recorded. You can share your screen by [direction for sharing].

Next, I would like you to visit the URL I sent you earlier today. To start, please look at this page and tell me what you make of it:

What are your first impressions?
What is this site for? You can scroll if you want to, but do not click on anything just yet.

Task completion

Thanks. Now I am going to ask you to respond to the following few scenarios. I am going to read each scenario out loud, and I will also share the text of the scenario via chat.

Example scenarios

Example scenario 1

You are building a training application that stores personal information. Show me how you would determine whether you need a privacy impact assessment.

Example scenario 2

You want to apply for a job with the federal government. Show me how you would go about doing that.

Example scenario 3

You want to check the status of your Social Security benefits. Show me how you would do that.

Follow-up

We are finished with the bulk of the test. You mentioned [something they said out loud] earlier and I did not want to jump in at that time. Can you say more about that? What are your thoughts about these concepts, generally?

Wrap-up

Thank you so much for your time. Before we finish:

What did we forget to ask about?
If you had a magic wand and could instantly change one thing about this website, what would it be? Why?
Would you be open to speaking with us again in the future?
Do you have any questions for us?
Who else should we talk to?

Best practices

For more best practices on how to conduct research and encourage collaboration, see the User research and collaboration guide series.

For a sample script and example questions for in-depth user and stakeholder interviews, see How to conduct an interview.

A list of indicators for usability test success

The following indicators can help determine if a usability test will produce useful results.

This is not an exhaustive list, but it can be helpful in planning and assessing usability tests.

Study design
Indicator	Try this	Instead of this
Study purpose	Researcher clearly articulates the purpose of the usability test (for example, as a specific question to answer or an area of inquiry).	Researcher does not specify a purpose for the test, or the purpose specified is very broad, like “testing the app” or “finding problems.”
Number of participants	Tests include a sufficient number of participants necessary to see patterns, and there is only one participant per session.	Study includes only one participant, or includes more than two participants testing simultaneously (like a focus group).
Sampling bias	Participants are a diverse set of people who actually use the application (or do the task the application is supposed to support).	Participants are experts (not average users), friends, family, or close colleagues of the product team.

Moderator style
Indicator	Try this	Instead of this
Question style	Researcher asks follow-up questions that are open-ended and non-leading.	Questions are leading or subtly suggest potential solutions or hypotheses.
Task variability	Researcher asks participants to complete similar tasks related to the study’s purpose.	Researcher asks participants to complete different tasks, or tasks unrelated to the study’s purpose.
Framing	Researcher asks participants to complete tasks with the product or service that align with their goals.	Researcher asks participants to complete tasks unrelated to their goals. For example, asking a participant how they might send a fax when their job does not call for that.
Priming	Moderator asks participants to complete tasks without indicating how that might be done. For example, “How would you view the status of your application?”	Researcher guides participants in completing tasks. For example, “Which of the links in the header would you click to login?”

Team participation
Indicator	Try this	Instead of this
Number of teammates	The team designates a single moderator for the test, and at least one member of the product team observes the usability test.	A single person from the product team participates in and leads the test.
Observer role	Observers do not moderate. They are generally quiet, and ask open-ended questions after the test has concluded.	Observers interrupt the participant, or attempt to sell or explain the product. Observers debate the participant’s actions in their presence.

Analyzing findings
Indicator	Try this	Instead of this
Notetaking	Tests are recorded or notes are taken for sharing with absent stakeholders.	Tests are not recorded, or test results are not documented.
Synthesis	Moderator debriefs with teammates immediately after each interview. Researcher looks for patterns across multiple participants and surfaces problems that affected several people.	Moderator reports the most memorable problems without identifying patterns across sessions.
Presentation of findings	Researcher reports findings to team and stakeholders in an easy to follow, well prioritized way.	Researcher presents the team a “basket of issues” or an unprioritized laundry list of potential changes.
Incorporation of findings	Product team translates findings into future user stories or product refinements.	Researcher reports do not affect the product backlog or ongoing development work.