Thursday, August 29, 2013

With usability, everyone wins

Having demonstrated a usability test in my last post, where do we go from here? I hope I"ve shown usability should not be just tacked onto a project. Usability needs to be part of the design of open source software, and addressed as a process. As open source software developers, we are generally very good at applying good software development practices to our work. Now we need to take the next step and bring usability into that practice.

Our next challenge in open source software is to find ways to incorporate usability into our development culture. This is a big step. Most open source programs are written by developers for other developers. Developing features takes priority, and we rarely look at how users will try to access that functionality.

In open source software, the community plays a strong role in testing new releases. Unfortunately, left on their own with no structure to usability testing, open source software testers will respond with simple bug reports: “This feature is confusing.” That's not helpful feedback. Or, usability bugs may not be afforded the same status as functionality bugs.

The approach to identify usability issues in open source software, therefore, needs to be more structured. Open source software developers can apply a variety of methods, although the ideal would be to conduct formal usability tests at every major release.

Usability testing for open source software projects doesn’t need to be performed in a lab; a project can find ways to “crowdsource” usability testing with the user community. For example, the open source web content management system Drupal streamed usability testers as they undertook a usability test. This allowed Drupal developers all over the world to observe the usability test without having to travel to a single location. When open source developers can watch testers experience problems with the software, they better understand the issues and how to address them.

Another simple method is usability testing by “flash mob.” Just intercept people in a public space and ask them to try out a few scenarios and share their experiences. If your test subject is willing to spare a few minutes, and with a “mob” of individual testers and evaluators, you can glean valuable information in a short amount of time. This idea of “flash mob” usability testing can be extended to other domains, too. If your open source software is web-based, you might conduct similar impromptu usability tests by intercepting website visitors.

You don't have to be an expert to apply usability tests in open source software. Anyone can do it. You only need to watch users try to use the software, and usability issues will immediately become obvious. As few as five testers will give you consistent information that you can use to make your open source software even easier to use. With good usability, everyone wins.

Wednesday, August 28, 2013

My own usability test

To demonstrate the formal usability test, I reviewed three common open source software projects. I solicited advice here and on other forums, asking which open source software programs to test. Sorting through the suggestions, three open source software projects met the criteria for the usability test:

  1. Gedit (a text editor)
  2. Firefox (a web browser)
  3. Nautilus (a file manager)


I didn't ask for a specific level of technological expertise, as I was looking for typical users with average knowledge. In most formal usability tests, it’s common to give each tester a small gratuity; I gave out free pop and candy.

Seven people volunteered for my usability test. They ranged in age from about 20 to about 40, three men and four women. Most claimed “low” experience with computers, and almost all used Windows as their primary desktop. Each tester started with a fresh instance to test against: their own account on a laptop running Fedora Linux. Since everyone had a separate account, they started from the same “default” settings.

At the start of the usability test, I gave each tester a brief background. I explained that this was a usability test, so it was about the software, not about them. If the tester experienced problems during the test, that was okay and we could move on to the next section. I was there to observe their interaction with the software, not to judge. I would take notes and watch what was happening on their screen.

I also asked each tester to speak aloud what was going through their mind during the usability test. For example, if they were looking for a Print button, they should simply say, “I’m looking for a Print button.” And I encouraged them to track the mouse on the screen with their eyes so I could observe where they were trying to find menus and buttons.

During the usability test, I presented the testers with a number of scenarios, each providing a brief context and an action they were asked to complete. For example, after asking testers to navigate to the BBC News website in the Firefox browser, one scenario asked them to increase the font size for the site. Importantly, the scenario does not use the same wording as the menu action that applies the font change:
You don't have your glasses with you, so it's hard to read the text on the BBC News website. Please make the text bigger on the BBC News website.

Overall, the usability test included nineteen scenarios, which testers completed in 30-40 minutes.

So, what were the usability issues? Interestingly, almost everyone experienced the same four issues:

  1. Change the default font in Gedit.
  2. Change the default colors in Gedit.
  3. Create a bookmark or shortcut to a folder in Nautilus.
  4. Search for a file in Nautilus.

In Gedit, testers were very confused about how to set the default font and colors. Part of this confusion stemmed from thinking of the editor as if it were a word processor such as Microsoft Word, which uses icons on the toolbar to accomplish either action.

In Nautilus, testers became frustrated while trying to create a bookmark or shortcut to a folder (used for a project that collected photos, and was saved under the Pictures folder). The only user who successfully created the bookmark only did so by accident.

Similarly, most testers found searching for a file in Nautilus a difficult task. They did not realize that the Search function starts from the current folder, not from their Home directory. Only two testers were able to use Search successfully. Of these, one happened to click on the Search button from their home directory. The other tried changing options in the drop-down Search action until they eventually picked a combination that worked. One tester gave up with Search and simply navigated into each folder in turn to find the file they were looking for.

And while GNOME was not part of the usability test, almost all testers experienced difficulty with the GNOME “Activities” menu hot corner. In the GNOME desktop environment, the “Activities” menu shows the list of available programs plus a view of the running applications. Users can bring up the “Activities” by clicking the menu icon in the upper left corner of the desktop, or simply by moving the mouse into that corner (the “hot corner”). Usually right away in the first scenario, testers would “overshoot” the program menu they were looking for, and hit the GNOME hot corner. Although testers were able to recover from the hot corner, it caused frequent disruption throughout the test.

What worked well for usability? Throughout the test, I observed four themes of good usability that allowed all testers to quickly pass through those parts of the usability test:

1. Familiarity
Testers commented that the programs seemed to operate more or less like their counterparts in Windows or MacOS X. For example, Gedit isn't very different from Windows Notepad, or even Microsoft Word. Firefox looks like other web browsers. Nautilus is quite similar to Windows Explorer or MacOS X Finder. To some extent, these testers had been “trained” under Windows or MacOS X, so having functionality (and paths to that functionality) that was approximately equivalent to the Windows or MacOS X experience was an important part of their success.

2. Consistency
User interface consistency between the programs worked strongly in favor of the testers, and was a recurring theme for good usability. Right-click worked in all programs to bring up a context-sensitive menu. Programs looked and acted the same, so testers didn't have to “re-learn” how to use the next program. While the toolbars differed, all programs shared a familiar menu system that featured File, Edit, View, and Help.

3. Menus
Testers preferred to access the programs’ functionality from the menus rather than via “hotkeys” or icons on the toolbar. For example, the only toolbar icon that testers used in the Gedit scenarios was the Save button. For other scenarios, testers used the drop-down menus such as File, Edit, View, and Tools.

4. Obviousness
When an action produced an obvious result, or clearly indicated success—such as saving a file in the editor, creating a folder in the file manager, opening a new tab in the web browser—testers were able to quickly move through the scenarios. Where an action did not produce obvious feedback, the testers tended to become confused. The contrast was evident when trying to create a bookmark or shortcut in the Nautilus file manager. In this case, Nautilus did not provide feedback, failing to indicate whether or not the bookmark had been created, so testers were unsure if they had successfully completed the activity.

These are good lessons in open source software and usability. Your program's user interface doesn't have to be a beautiful impediment to understanding. Instead, leverage existing user interface paradigms: Be consistent with other programs on the same platform, whether other open source software or proprietary software. Use menus that are clearly labelled. Ensure every action has a result that is obvious to the end user, especially if that result indicates failure.

Monday, August 26, 2013

Applying usability tests to your own programs

Let's walk through a usability test, to show how open source software developers might apply this method to their own programs. Remember, the purpose of a usability test is to uncover issues that general users have in using the program. This is a practical test of its ease of operation, and as such differs from quality assurance or unit testing.

To start, define a set of written scenarios that represent how typical users with average knowledge would use the program. You aren't looking to test every feature of the program, just the tasks that most users would want to do. Make your scenarios realistic; provide a short description of each scenario and ask the tester to perform tasks within that context. Write scenarios that don't lead the tester by using the product's words for actions, especially if your typical users of average knowledge might not know those words.

For example, if you want to evaluate an editor, your scenarios might ask the tester to enter a short text document, save it, and make basic edits to the file. For a web browser, your scenarios could ask the user to search for a website, bookmark a site, or save a copy of a web page for offline use.

Invite testers for a usability test. You don't need very many people, five testers will give you great results. Give the tester one scenario at a time, each on a separate piece of paper, and ask them to complete the tasks. Observe what they do in the program: the routes they take to accomplish the tasks, and the problems they encounter. Take plenty of notes.

The most difficult part of a usability test is watching a tester struggle to locate a menu or button. While the right action may seem apparent to you, the value is in learning and identifying what is not obvious for a user. Do not give hints. If a tester is unable to finish a scenario, that’s okay; just move on to the next scenario.

At the end, ask followup questions of your tester. For example, you might ask “You seemed lost when you tried to do X, what would have made it easier?” Or perhaps “What were you expecting to see on the screen when you were doing Y?” As a wrap-up, ask the tester to describe what worked well in the program, and what features should be improved.

Friday, August 23, 2013

Applying Usability in Open Source Software

Open source software developers have created an array of amazing programs that provide a great working environment with rich functionality. At work and at home, I routinely run Fedora Linux on my desktop, using Firefox and LibreOffice for most of my daily tasks.

But as great as open source can be, we've all shared or heard the same comments about certain open source programs:

  • “___ is a great program … once you figure out how to use it.”
  • “You can do a lot in ___ … after you get past the awkward menus.”
  • “You’ll like using ___ … if you can learn the user interface.”

That’s the problem, right there. People shouldn't have to figure out how to use a program, aside from specialized tools. Typical users with average knowledge should be able to operate most programs. If a program is too hard to use, the problem is more likely with the program than with the user.

Most open source programs are written by developers for other developers. While some large open source programs such as GNOME and Drupal have undergone usability studies, most projects lack the resources or interest to pursue a usability evaluation. As a result, open source software programs are often utilitarian, focused on the core functionality and features, with little attention paid to the user interface.

So, how can open source developers easily apply usability to their own programs? It depends. Typical methods run the gamut from interviews and focus groups to heuristic reviews and formal usability tests.

However, such formal practices are often antithetical the open source software community. Open source developers mostly prefer functionality over appearance. While some projects may have a maintainer who dictates a particular design preference, most programmers view menus and icons as “eye candy.”

With that in mind, open source software developers should consider the usability methods that apply well to the culture of open source software development. I propose this list:

Heuristic review
An experienced usability expert reviews a program design, and presents a marked-up list back to the developer with suggestions for improvement.
Prototype test
Ask testers to evaluate a mock-up of a program design, even if that design is just screenshots on paper.
Formal usability test
Testers use the program according to a set of scenarios, and an observer notes where they have problems.
Questionnaires
Testers use a program against a set of scenarios, then fill out a questionnaire about what worked well and what should be be improved.

You don’t need years of usability experience to apply usability in open source software development. Developers can learn a lot just by sitting down with a few users and watch them use the software.

Whatever method you choose, the value of usability testing is doing it during development—not after the fact. Apply usability testing a few times; at each round of testing, you will have identified a number of issues that you can fix, and further improve your project.

Thursday, August 22, 2013

A parallel analysis via Molly Bang

I'd like to follow up on the user interface analysis via Molly Bang by way of analyzing a static image using Bang's methods. Bang’s method is well suited for interpreting imagery, providing a basis to evaluate shapes and patterns within an image.

For simplicity, let's analyze a webcomic that may be familiar to many of you: “Piled Higher and Deeper” (PHD) is a webcomic about life in academia. The artist and author, Jorge Cham, earned his PhD in Mechanical Engineering at Stanford University, and later became a full-time instructor and researcher at the California Institute of Technology from 2003-2005. Cham now works on the comic full-time.

The PHD comic website claims over fourteen million monthly pageviews and over one million unique monthly visitors. Readers to the PHD website represent over eight hundred different fields of study from over one thousand different schools worldwide. Comics get their topics out of the socio-cultural environment from which they originate and for which they are produced. PHD reaches a mostly internal audience of fellow academics and offers a rhetorical view about life in academia. Rhetoric is the set or system of principles regarding the functional application of conventional signs for the purpose of getting someone to think something or want to do something. The comics in PHD do not advocate a specific action, but do evoke an exigence to awaken its readers from complacency, stirring its audience out of a neutral state.

PHD comic #1436 The Evolution of Intellectual Freedom originally published 20 July, 2011, describes research choices at different times in an academic career. Ostensibly organized in a two dimensional chart, the webcomic represents research preferences for undergraduate students, graduate students, assistant professors, tenured professors, and emeriti professors. At each moment, a character expresses their optimism or pessimism towards research options; the undergraduate student and emeritus professor both claim they will “research whatever I want” while the graduate student, assistant professor, and tenured professor assert they will perform the research desired by others, such as the assistant professor’s tenure committee.


“The Evolution of Intellectual Freedom” presents these six moments in time without panels or frames. In comics, panels typically fracture both time and space, and gutters between panels allow the reader to connect these moments to mentally construct a continuous, unified reality. Instead, “The Evolution of Intellectual Freedom” uses the two dimensional chart as an outer frame to infer the passage of time, referencing graphs where a variable (on the vertical axis) is visualized over time (on the horizontal axis). Gutters are implied by the spaces separating the characters. The comic uses these scene-to-scene transitions to transport the reader across time in an academic career.

Now let's go back to Bang for the analysis. Molly Bang's book, Picture This, illustrates ten basic principles present in powerful pictures: Smooth, flat, horizontal shapes give us a sense of stability and calm. Vertical shapes are more exciting and more active; they imply energy. Diagonal shapes are dynamic because they imply motion or tension. The upper half of a picture is a place of freedom, happiness, and triumph, while the bottom half of a picture feels more threatened, heavier, sadder, or constrained; objects placed in the bottom half also feel more grounded. The center of the page is the most effective “center of attention.” It is the point of greatest attraction. White or light backgrounds feel safer than dark backgrounds. Pointed shapes are threatening, while rounded shapes or curves are secure or comforting. Larger object in a picture are perceived as stronger. Audiences associate the same or similar colors much more strongly than the same or similar shapes. Contrasts enable perception of patterns and elements.

Applying Bang’s method does not require all ten basic principles to examine “The Evolution of Intellectual Freedom.” Applying only a few principles permits interpretation of the image:

Diagonal shapes imply motion or tension.
The graduate student, assistant professor, and tenured professor are drawn using diagonal lines. The graduate student stoops over a computer terminal, with both his body and the monitor tilted on a diagonal. The assistant professor is depicted running, using diagonal lines to draw arms and legs. The tenured professor, while seated, suggests tension through an arched body and crossed legs, both on a diagonal. Even the emeritus professor and tombstone are drawn using diagonal lines, implying tension without motion.
The undergraduate includes some diagonal lines in his left arm posture, but otherwise is drawn using only vertical and horizontal lines. This is the only character who is without tension.

The upper half of a picture is a place of freedom, happiness, and triumph, while the bottom half of a picture feels more threatened, heavier, sadder, or constrained.
“The Evolution of Intellectual Freedom” uses the two dimensional chart as an outer frame. All characters are drawn on the bottom of the image, suggesting these figures are sadder or more constrained. The graduate student, assistant professor, and tenured professor reflect this in claiming their research interests are controlled by outside interests.

Audiences associate the same or similar colors much more strongly than the same or similar shapes.
No two characters in the comic share the same or similar colors. However, the emeritus professor and tombstone are both colored grey, suggesting the grave belongs to the emeritus professor. The relationship is further implied by the interruption in the emeritus professor’s statement, which connotes that the emeritus professor expired before he was able to conduct research according to his own agenda.
While audiences associate the same or similar shapes less strongly than the same or similar colors, shape does imply relationships between objects. The undergraduate and the emeritus professor are both drawn in the same pose: right arm extended and pointing upwards with a single extended digit, elongated round mouth, arched eyebrows, and pointed nose. This evidence suggests the two characters are the same person, young and old, before and after his professional career.

Sunday, August 18, 2013

User interface analysis via Molly Bang

I'd like to share some brief thoughts about how one might discuss user interfaces. Analysis about what "works" in user interface design might come from any number of sources. Jakob Nielsen is one obvious reference; Nielsen is an author, researcher, and consultant on user interfaces, especially Web usability, Web design strategy, and user-centered methodology. Janice (Ginny) Redish is another resource; Redish is a recognized specialist in plain language writing for the web user experience (UX) research, and consultant specialist in usability.

However, we may also base an analytical discussion of imagery on the work of American illustrator Molly Bang. Her book, Picture This, illustrates ten basic principles present in powerful pictures:
  1. Smooth, flat, horizontal shapes give us a sense of stability and calm.
  2. Vertical shapes are more exciting and more active; they imply energy.
  3. Diagonal shapes are dynamic because they imply motion or tension.
  4. The upper half of a picture is a place of freedom, happiness, and triumph, while the bottom half of a picture feels more threatened, heavier, sadder, or constrained; objects placed in the bottom half also feel more grounded.
  5. The center of the page is the most effective “center of attention.” It is the point of greatest attraction.
  6. White or light backgrounds feel safer than dark backgrounds.
  7. Pointed shapes are threatening, while rounded shapes or curves are secure or comforting.
  8. Larger object in a picture are perceived as stronger.
  9. Audiences associate the same or similar colors much more strongly than the same or similar shapes.
  10. Contrasts enable perception of patterns and elements.
(Bang, Molly. Picture This: How Pictures Work. San Francisco: Chronicle Books, 2000.)

Let me take a moment to briefly explore a few user interface concepts using Bang's principles:

Bang's principle that the "upper half of a picture is a place of freedom, happiness, and triumph" while "objects placed in the bottom half feel more grounded" (4) seems to be mirrored in the Mac interface vs the Windows interface. MacOSX places the "menu bar" at the top of the screen: File, Edit, View, etc. while both MacOSX and Windows locate the "action bar" at the bottom (MacOSX calls this the "Dock" while Windows labels it the "Taskbar," but both do essentially the same thing: represent currently-running programs, and provide a menu to start new programs). Mac users may relate to adjectives such as "freedom, happiness, and triumph" - and if they've ever wondered why, it may be because their program interaction is at the very top of the screen. GNOME 3 puts the "Activities" menu at the top of the screen - but it uses a black background (see next).

The idea that "light backgrounds feel safer than dark backgrounds" (6) is important in understanding why I decided to switch my "Aura-like" Xfce desktop from a dark theme to a light theme. I mentioned the darker colors made things more difficult to read, and the light grey theme ("Adwaita") provided much better visibility. But it was also a matter of not "feeling" right. With the dark theme, I sometimes felt lost. Changing to the light grey theme immediately changed my perception; things seemed easier to use, even when I had changed nothing other than the color. Interestingly, the black "Activities" menu bar in GNOME 3 may contribute to users' negative reaction.

That "rounded shapes or curves are secure or comforting" (7) may explain the recent (last five years?) trend with user interfaces to provide rounded edges. Of course, Apple's original Macintosh computer (1984) used rounded corners. Today, MacOSX and many Linux desktops (including GNOME 3) make use of rounded edges, both for the desktop and for program windows. Xfce and Microsoft Windows use "hard" corners for the desktop, but round corners for program windows.

Contrast (perception of patterns and elements, 10) and colors (associate the same or similar colors, 9) may be significant elements in successful user interfaces. I'll return to this idea in a later discussion.