imedia.ca | perception | design sourcebook for business

Usability Testing Basics

What does a usability test determine?

Usability testing answers questions about how people use your Web site. Testing looks at navigation, site look and feel (user interface), content, features and functionality, and shopping experience to determine what areas can be improved. Each area is tested for utility, learnability, efficiency, retainability, error handling, and satisfaction. Table 1 describes the six major testing attributes.

Why do I need a usability test?

As a site owner, you are very familiar with the information you want your site to contain and in your own mind, familiar with how it goes together. The catch is, "in your own mind." Other people don't necessarily think like you do, and the knowledge that makes you an asset to your company can be a blind spot when it comes to usability.

Usability testing lets you examine your Web site and process model from the outside. By learning how others want to use your site, you can learn how to best serve them. And that is what brings customers back.

Setting up a usability test

Create a survey.
Usability testing takes many forms, from five-person user surveys to hundred-person marketing surveys. In either case, before the definition of requirements, information about needs and objectives must be obtained.

Testers must be real users, representative of the target audience. Testers must perform real tasks. Survey a group of real users to determine the user group and to find the most commonly performed tasks. Questions that must be answered include:

Who are the users?
What is the user's goal?
How do they use it?
What information do they need?
Where do they use it? (PC/Mac/Wireless?)

After answering these questions, you can select a minimum of six people for testing purposes and develop a usability test that encompasses the key site features.

Beginning the testing process.
The first part of testing involves the user performing common actions on the Web site. These actions should be carefully observed and timed. Number of mouse-clicks should be noted, as well as the number of fields that need to be completed.

During the testing process.

Observe do not prompt.
Do not talk to the user during testing.
Watch the user independently perform the task.
Do not help or prompt the user.
Track the user's actions.
Track the time it takes the user to perform each task.

After the tasks are completed, have the user complete a detailed questionnaire about the assigned tasks. Typical questions may be:

How long did it take you to find the page to complete task X?
How many clicks did it take you to get there?
Do you feel that is too many clicks?
Once you started task X, how easy was it to complete the task?
How many pages did you have to visit?
Was the number of pages enough, too many, too little?

Analyzing the results.
After testing is complete, results will be collected and studied. Comparisons will be made between the users' perception of completion and actuality (e.g. they thought it took five clicks, when it actually took seven), and their general feelings about the performance of the site will be summarized.

Specific ratings will also be included, based on questions similar to the following:

The results will include statistics relevant to the testing. For example, when checking for over all look and feel, a result might be, "75% of users tested found that red links were distracting." Average access times and page loading speeds, gathered under various circumstances, will also be included in the results.

Sample Questions

Usability Testing: Navigation, Look and Feel, Content, and Features and Functionality

Utility

Did you find the visual design appealing?
Did you think the graphical look of the site reflects the products?
Was it obvious to you which site you were on by the visuals on the home page?
Was the navigation clearly labeled?
Was the navigation placed in an intuitively logical spot?
Average # of navigation items per page, was there enough choices, too many?
Are alternative forms of navigation provided? Are they clearly visible?
Was the navigation organized in a logical manner?
Can you tell where you are within the site from the page you are on?
How is this achieved? - URL, Page Label, Selected/deselected labels, Depth gauge, Colour, Design Style, Not clearly identified
Were you able to locate the site map?
Did you fine the site map useful?
Is there a date display on the site? If Yes, did you find it useful?
Do you find personal site customization useful? (i.e. msn.com personalized content/layout feature) Give reasons.
Were you able to go back to what you were previously doing before entering the shopping cart area?
Could you find more information on your topic somewhere else on the site when you needed it?

Learnability

Does the site have immediate clues as to how to use it? What are they?
Was it similar in look to any other sites you've used?
Does the visual design help indicate areas of importance to you?
Were headings on information clear and relevant?
Were section names reflective of the content found in them?
Did you find the available navigation effective in guiding you through the web site?
Identify actionable/clickable elements of the web site by circling areas on a printout of the home page.
Could you predict what type of information you would find behind a navigation link?
Does the navigation look pressable? How is this achieved?
Are link colours modified from the default blue?
If yes, did you find this confusing?
What is your initial learning time?
If membership is offered, is it easy to register to become a member?

Efficiency

Did the graphics load quickly enough for you to be able to locate information right away? If not, were "alt" tags visible to guide you?
Did you find any of the graphics/animations distracting, or getting in the way of finding information?
Is the information clear and easily read?
Could you find what you were looking for easily and quickly?
How difficult was it? 1 to 10
Is there a browse by category feature? If Yes, how useful did you find this?
Is there a search function? If yes, did you feel the need to use it?
Did you get useful search results?
Is there an "advanced search" feature available? If Yes, did it find the information much better than the standard search?
Is it easy to add products to the shopping cart? If No, why? (Location, too many clicks etc·)
Were you able to change the item quantity easily?
Was the check out located conveniently? If No, where would you suggest it being placed?
Did you find your account easily updateable?
Did you find the number of steps to complete a transaction excessive?
Did you feel the need to use the back and forward buttons?
Did you feel the need for a site map?
Do you know what cookies are? Do you ever disable cookie support? If Yes, give reasons.

Retainability

Is there an option to remember your Username? If Yes, did you use this feature?
Would you prefer an option to remember both your Username and Password?
Was the available navigation consistent throughout the site?
Were you able to carry over any learned behaviours throughout the site?

Errors

Do you generally use the help feature? If Yes, what exactly do you seek help for? (Site navigation, FAQ's, Customer Service etc·)
Did the help section offer the answers you were looking for?
Is there a way to retrieve a forgotten password? If Yes, were the instructions clear to you?
Was there a way indicated to recover from an error?
Did you lose all your information entered in fields?

Satisfaction

What features do you feel are missing and would like to see on this site?
Are there clear membership benefits?
Do you often subscribe to newsletters/mailing lists? If No, why not?
Are there clear benefits of why you're subscribing?
After a purchase is there a clear confirmation?
Is there a way to track the status of your order?
Was there an option to call in for assistance?
How would you rate the overall experience of using the site? 1(least pleasant or positive?) to 10(most pleasant)
Does the look of the site affect your confidence in the brand?
Do you prefer a Web site that has a distinct graphical look or has a more simple, text-based design?
Do you like the colour scheme used on this site?
Did you find the information believable(credibility)
Would you visit this site again to buy/find information on a product?(loyalty)
What complications did you encounter while completing the purchase?

Usability Testing: Shopping Experience