Saturday, November 17, 2012

A preview of usability test results

In my discussions with thought leaders about usability testing, I learned that you can often get a surprisingly accurate picture of usability in as few as 3 testers. It's one thing to hear that said; it's another to see it happen.

This week, I worked with 7 usability testers in an informal lab setting: I set up a laptop in my office, with a USB keyboard and mouse, booted into a USB fob drive installation of Fedora 17 running the Gnome desktop. Each tester was logged into their own account, thus each tester started from the same "default" state.

Testers ranged from about age 20 to mid-40s, both male (3) and female (4). Most testers (5) claimed "low" experience with computers, and almost all (6) used Windows as their primary desktop.

I'll do a more detailed write-up of results soon, but here's a preview of the findings:


Testers were easily able to accomplish the following tasks:

  • Write a simple note. The scenario asked them to type in a short note (about two paragraphs) so I could observe them recovering from typos and errors.
  • Basic editing. Replacing a single instance of a few words, and move a sentence from one paragraph to the other.
  • Replacing text. This asked testers to replace all occurrences of two names with new spellings of the same names.
  • Saving a file as a new copy of the file, with a new name.

However, they experienced great difficulty in these tasks:

  • Changing the default font. Everyone agreed that an editor, even a simple editor like Windows Notepad (and Gedit is similar in function to Notepad), should let them change the text.
  • Changing the default colors. Most said that they would expect to change the colors, but either had trouble finding this function or were were unable to find it altogether.


With a few one-off exceptions, testers had no problems using Firefox to achieve these tasks:

  • Search the web for a particular news site, and navigate to that site.
  • Set that website as the browser's default startup page.
  • Increase the font size.
  • Create a new tab or window, and navigate to a particular URL.
  • Save the page for off-line viewing.
  • Download an image from the website.
  • Make a bookmark to a website for later use, and give the bookmark a new name.


Testers were easily able to complete these tasks:

  • Create a new folder.
  • Move a folder to a new location.
  • Give a folder a new name.
  • Delete a file.

However, testers experienced great difficulty with these tasks:

  • Create a new bookmark or shortcut for a folder.
  • Search for a file.

I'll dig into these results further in another post.

No comments:

Post a Comment