Saturday, February 8, 2014

Looking ahead to my research

Thinking back about my research, I wanted to look ahead to how to make the analysis applicable to practice.

My research question is along the lines of "How well do users navigate/understand/… the new design patterns in GNOME?" A practical approach to this research question would be to measure the success/impact of these design patterns on typical behavior/activities. And usability can be measured; it is a quantitative practice. One analytical method is to generate a "heat map" that demonstrates the relative ease or difficulty in accomplishing certain scenarios or actions.

One example of a heat map to show usability test results:


Gedit
█ █ █ █ █ █ █
G1. Type a sample note (provided) and save the file.

█ █ █ █ █ █ █
G2. Edit text within the note.

█ █ █ █ █ █ █
G3. Replace all instances of several words.

█ █ █ █ █ █ █
G4. Save the file under a new name.

█ █ █ █ █ █
G5. Change the default font.

█ █ █ █ █ █ █
G6. Change the default colors.


Firefox
█ █ █ █ █ █ █
F1. Search for and navigate to the BBC News website.

█ █ █ █ █ █
F2. Set the website as the default page.

█ █ █ █ █ █ █
F3. Increase the font size.

█ █ █ █ █ █ █
F4. Create a new tab and navigate to www.freedos.org.

█ █ █ █ █ █ █
F5. Save a copy of a web page for offline use.

█ █ █ █ █ █
F6. Download an image from the website.

█ █ █ █ █ █ █
F7. Create a bookmark to the website.

Nautilus
█ █ █ █ █ █ █
N1. Create a folder.

█ █ █ █ █ █ █
N2. Move the folder to a new location.

█ █ █ █ █ █ █
N3. Rename the folder.

█ █ █ █ █ █
N4. Create a bookmark or shortcut to a folder.

█ █ █ █ █
N5. Delete a file.

█ █ █ █
N6. Search for a file.
Source: “It’s about the user: Applying usability in open source software.” Jim Hall. Linux Journal, December 2013.


I find the heat map to be a useful tool to quickly visualize usability test results. Each colored square represents an individual test participant. The color represents the relative "success" in the tester accomplishing the test scenario: green (completed with little or no difficulty), orange (completed with difficulty), or red (unable to complete). You can see in the above example, four test scenarios were most difficult for the seven usability testers: G5, G6, N4, and N6. While a few test participants experienced difficulty in some of the other scenarios (F2, F6, and N5) these were isolated cases. The heat map provides quantitative results to usability analysis.

One possible improvement to this method would be to borrow from a common rhetorical analysis technique (see chapter 4, Hart, Roderick P., and Daughton, Suzanne M. Modern Rhetorical Criticism. Boston: Pearson Education, 2005) where a team of usability test observers rate the relative success of each test scenario, possibly on a four-color scale: green (no difficulty), yellow (some difficulty), orange (great difficulty), red (unable to complete).

No comments:

Post a Comment