Friday, February 14, 2014

An example of web usability

While this blog is about open source software & usability, I am interested in usability of all kinds, wherever it occurs. So I was very interested to read this announcement about how the U of M's Usability Services helped improve an online donation form. I assume it's okay to repeat the email here:
The University of Minnesota Foundation (UMF) raises and manages private dollars to advance the University’s mission. In an effort to boost online giving, the Foundation engaged IT’s Usability Services to discover ways they might improve their online giving tool to make giving easier for donors. Usability consultants performed a comparative user experience test of the previous donation tools and then two rounds of user experience testing on the newly developed tool in both mobile and desktop formats. These activities helped the team evaluate terminology, navigation, user expectations, and task flow. Based on on what was revealed in the assessments, UMF made significant changes their online giving tool.

After “Give to the Max” day, one of the biggest online giving days of the year, Foundation Systems Analyst Peter Dahl was thrilled with the results, stating, “On one of the biggest online giving days of the year, our redesigned giving tool outperformed the previous one on many measures.”
  • 34% more people gave a gift this year vs. last year
  • Last year 62% of people abandoned immediately when they saw the tool; this year only 33% abandoned immediately
  • Of 1,450 visitors, 23% converted to making a gift (vs. 17% at Amazon)
  • Last year 53 mobile visitors, 0 gifts; this year 200 mobile visitors, 27 gifts
These truly amazing, tangible results demonstrate the positive impact of good design based on understanding user behaviors and needs. The dollars raised via the improved interface will fund scholarships, world-class faculty, leading-edge research, new facilities, and academic programs on all five campuses of the University of Minnesota.

If you’re interested in increasing the effectiveness of your web presence, contact Usability Services.

Usability is a quantitative process, and you can measure the results. Here you can see the information product resulted in fewer people dropping out of the donation site, leading more people to donate. Additionally, the new website is mobile friendly, and 27 people donated through the mobile site (none did so last year).

Saturday, February 8, 2014

Who are the target users?

In a guest post on SourceForge.net, I wrote about Usability in Open Source Software, where I shared three essential steps to do a usability test:

1. Figure out who are the target users for your program.
You probably already know this. Are you writing a program for general users with average computer knowledge? Or are you writing a specialized tool that will be used by experts? Take an honest look. For example, one open source software project I work on is the FreeDOS Project, and we figured out long ago that it wasn't just DOS experts who wanted to use FreeDOS. We determined there were three different types of users: people who want to use FreeDOS to play DOS games, people who need to use FreeDOS to run work applications, and developers who need FreeDOS to create embedded systems.

2. Identify the typical tasks for your users.
What will these people use your program for? How will your users try to use your program? Come up with a list of typical activities that people would actually do. Don’t try to think of the steps they will use in the program, just the types of activities. For example, if you were working on a web browser, you’d want to list things like: visiting a website, bookmarking a website, or printing a web page.

3. Use these tasks to build a usability test scenario.
Write up each task in plain language that everyone can understand, with each task on its own page. Put a typical scenario behind the tasks, so that each step seems natural. You don’t have to build on each step – each task can be separate with its own scenario. But here’s the tricky part: be careful not to use terminology from your program in the scenario. Also avoid abbreviations, even if they seem common to you – they might not be common to someone else. For example, if you were writing a scenario for a simple text editor, and the editor has a “Font” menu, try to write your scenario without using the word “Font.” For example, instead of this: “To see the text more clearly, you decide to change the font size. Increase the font size to 14pt.” write your scenario like this: “To see the text more clearly, you decide to make the text bigger. Increase the size of the text to 14 point.”

In my research question, I propose investigating the new design patterns in GNOME, asking "How well do users navigate/understand/… the new design patterns in GNOME?" A practical approach to this research question would be to measure the success/impact of these design patterns on typical behavior/activities. And usability can be measured; it is a quantitative practice. One way to explore the research question would be to conduct a usability test against prototypes (functional or on-paper) of the updated applications, using the new design patterns. By testing several applications that use the same or similar design patterns, the results might identify areas where the design patterns work well vs where they do not work well.

So my first step is to determine who are the target users for GNOME. From that, I can then decide the applications that should be tested. GNOME designer Allan Day suggested a list of GNOME applications that use the new design patterns concept. I might use any of these in my usability test:

(see also the full Applications list)

While these applications are all good candidates for a usability test, the target users may define which applications (and thus, design patterns) would make best candidates for a usability test.

A few useful links

Allan Day of GNOME shared a few links with me about usability, which I wanted to share here for reference:

GNOME Human Interface Guidelines (HIG)
GNOME is currently documenting the design patterns used in GNOME 3 applications on the HIG wiki. This is very much work in progress. An updated Human Interface Guidelines document focusing on GNOME 3 will be released in due course.(ref)

Supporting the user experience in free/libre/open source software development (PDF)
PhD thesis by Paula M. Bach (2009, Pennsylvania State University). 209 pages.

Usability in Free Software
A guide by Jan-Christoph Borchardt. From the abstract: "For a software to truly be free, people need to be able to easily use it without help. This is a quick start to usability testing in a distributed and independent development environment: A collection of usability methods applicable especially in free & open source software projects. No matter if you are a developer, usability person or just enthusiastic about making software easier to use, this document should help you get going."

How to not report your UX bug
Presentation (45 min) by Fabiana Simoes at GUADEC 2013 (GNOME Users’ And Developers’ European Conference, held annually in cities around Europe). From the abstract: "Empowering users to proactively contribute to FOSS projects by reporting UX issues is not only a way to leverage UX in FOSS projects, but one that fits the FOSS community values. By referring to the users' personal experiences, designers in FOSS communities may build and inform their design decisions in an open and transparent manner. Not only that, but this is also a way by which we can engage our users in the values embedded in our community. For this to be possible, however, the information provided by users in their reports and the information designers need have to be aligned. DOs and DON'Ts for reporting UX bugs that are actually useful for designers and feasible for them to act upon."

Areas of investigation

The GNOME Human Interface Guidelines (HIG) establish a set of application design patterns common to the GNOME environment. From the HIG wiki: "Design patterns are the main elements that make up an application design. Some patterns, like application menus or header bars, are essential. Others are optional. Patterns can also be combined. Making appropriate choices about which design patterns to use is an important part in designing a GNOME application." (emphasis mine)

These design patterns in GNOME 3 are quite different from GNOME 2. And as GNOME continues to grow, the project also continues to change and improve these design patterns. For example, GNOME 3 applications use a "header bar" in place of a traditional "title bar." The idea seems to be that the application title appears in GNOME's black "top bar," which becomes part of the application menu.

In GNOME, these patterns include:

  • Application menus (1)
  • Header bars (2)
  • View switcher (3)
  • Content overview
  • Grids
  • Lists
  • Search (4)
  • Selection mode  (5)
  • Action bars
  • Gear menu (6)
  • Sidebar list (7)
  • Notifications
  • Secondary windows
  • Popovers

(numbers correlate to diagram, from HIG wiki)


You can see an example of several such design patterns in this GNOME 3.10 screenshot (from Fedora 20):


In this screenshot, note the use of sidebar lists, navigation, search, gear menu, and view switchers in both GNOME applications.

One way to explore my research question would be to conduct a usability test against prototypes (functional or on-paper) of the new design patterns. By testing several applications that use the same or similar design patterns, the usability test results might identify usage areas where the design patterns work well vs where they do not work well. These results might inform future GNOME design and development - especially important, since "Making appropriate choices about which design patterns to use is an important part in designing a GNOME application" (HIG wiki).

Looking ahead to my research

Thinking back about my research, I wanted to look ahead to how to make the analysis applicable to practice.

My research question is along the lines of "How well do users navigate/understand/… the new design patterns in GNOME?" A practical approach to this research question would be to measure the success/impact of these design patterns on typical behavior/activities. And usability can be measured; it is a quantitative practice. One analytical method is to generate a "heat map" that demonstrates the relative ease or difficulty in accomplishing certain scenarios or actions.

One example of a heat map to show usability test results:


Gedit
█ █ █ █ █ █ █
G1. Type a sample note (provided) and save the file.

█ █ █ █ █ █ █
G2. Edit text within the note.

█ █ █ █ █ █ █
G3. Replace all instances of several words.

█ █ █ █ █ █ █
G4. Save the file under a new name.

█ █ █ █ █ █
G5. Change the default font.

█ █ █ █ █ █ █
G6. Change the default colors.


Firefox
█ █ █ █ █ █ █
F1. Search for and navigate to the BBC News website.

█ █ █ █ █ █
F2. Set the website as the default page.

█ █ █ █ █ █ █
F3. Increase the font size.

█ █ █ █ █ █ █
F4. Create a new tab and navigate to www.freedos.org.

█ █ █ █ █ █ █
F5. Save a copy of a web page for offline use.

█ █ █ █ █ █
F6. Download an image from the website.

█ █ █ █ █ █ █
F7. Create a bookmark to the website.

Nautilus
█ █ █ █ █ █ █
N1. Create a folder.

█ █ █ █ █ █ █
N2. Move the folder to a new location.

█ █ █ █ █ █ █
N3. Rename the folder.

█ █ █ █ █ █
N4. Create a bookmark or shortcut to a folder.

█ █ █ █ █
N5. Delete a file.

█ █ █ █
N6. Search for a file.
Source: “It’s about the user: Applying usability in open source software.” Jim Hall. Linux Journal, December 2013.


I find the heat map to be a useful tool to quickly visualize usability test results. Each colored square represents an individual test participant. The color represents the relative "success" in the tester accomplishing the test scenario: green (completed with little or no difficulty), orange (completed with difficulty), or red (unable to complete). You can see in the above example, four test scenarios were most difficult for the seven usability testers: G5, G6, N4, and N6. While a few test participants experienced difficulty in some of the other scenarios (F2, F6, and N5) these were isolated cases. The heat map provides quantitative results to usability analysis.

One possible improvement to this method would be to borrow from a common rhetorical analysis technique (see chapter 4, Hart, Roderick P., and Daughton, Suzanne M. Modern Rhetorical Criticism. Boston: Pearson Education, 2005) where a team of usability test observers rate the relative success of each test scenario, possibly on a four-color scale: green (no difficulty), yellow (some difficulty), orange (great difficulty), red (unable to complete).

Monday, February 3, 2014

If I could change two things

I've been thinking about what I would change in GNOME if I could change one thing. I find this an interesting exercise; what small change would have the biggest impact? I decided that I would move the "application icon" menu from the GNOME top bar to the application window. I believe that one change would improve the usability of GNOME, because users would immediately associate the "application icon" as a second menu. The menu remains affiliated with the application, in a way not currently possible when the "application icon" menu is part of the black top bar (which might be on a separate physical display from the application window).

I started to consider what other change I might make in GNOME to make it better. I began to consider the functionality that is currently "hidden" from the user, which might be exposed by tweaking the interface. My biggest concern: alerts and information.

In the current GNOME design, alerts and information messages are effectively "hidden" below the bottom of the screen. Many users—especially those new to GNOME—are unaware where to find these alerts and information messages. When the alert or message first occurs, they "pop up" from the bottom of the screen, only to disappear after a brief interval. How to find these messages again, if you didn't read them the first time? Perhaps you were away from your desk for a short time (on the phone, meeting with a colleague, taking notes on your office whiteboard, etc) and you weren't even aware the alert or information message had appeared on your screen. How would you know to look for them?

If I could change a second thing in GNOME, I would place "alert" and "information" icons in the top bar, probably towards the right side of the screen near the network, volume, and battery menu icons. To highlight them to the user, I would use a color for the icon, which would easily stand apart from the usual white-on-black color scheme in the top bar. Alerts and information messages are important to the user, and therefore more immediate; they should be readily apparent in the top bar, and free from the white-on-black icon standard.

I've picked up two such icons from Wikimedia Commons, both in the public domain: Achtung (for the alert icon) and Information Icon. Building on my previous mock-up, here's a concept drawing using the alert and information icons:

Mock-up of gedit, Nautilus, and Terminal; based on GNOME 3.10.3

I'm pleased with the appearance of the information message icon, but perhaps a different alert icon would be better. To suit the current icon scheme, I think the icons need to be limited to two colors, such as the white-on-blue information icon. Perhaps a white "!" on a red circle for the alert icon would be better, but I'll leave that to a graphics designer. This is enough to demonstrate my point.

(You may also notice that I re-added the "×" in the upper-right, to close the window. I mistakenly erased that in my previous mock-up, when I added a slight highlight to the active window.)

Imagine that you had briefly stepped away from your desk, or for some other reason your attention was not on your desktop. When you returned, would you immediately notice that GNOME had something to tell you? In this mock-up, you can clearly see that GNOME has at least one alert message, and at least one information message. If you click on the icon, you might get a drop-down box (similar to the "application icon" menu?) that lists a summary of each message.

Sunday, February 2, 2014

About my research

The focus of my M.S. capstone project will be a usability test of GNOME. This will expand on my previous directed study in usability, when I conducted a usability test of a few programs (gedit, Nautilus, Firefox) under GNOME 3.4 in Fedora 17. The connections to academic study are strong here; there has been very little academic research about usability in open source software. Nichols and Twidale previously researched "Usability Processes in Open Source Projects." And Benson, Müller-Prove, & Mzourek published their results of "Professional Usability in Open Source Projects: GNOME, OpenOffice.org, NetBeans." GNOME has conducted a few usability tests. Otherwise, research and academia have not largely explored this area of practice.

And as suggested by Andersen (2013), academia and practice need to develop a reciprocal relationship. It's a cycle; academia needs to provide accessible, actionable research that is published in places visited by practice. That allows the practice to advance, which provides future research opportunity for academia. The cycle continues, and both academia and practice benefit.

The question of the research question, necessary for research, also plays a role in this cycle. Rude (2009) describes how a research question arises from a rhetorical exigence (an urgency) and the anticipation of possible solutions. So in conducting research on usability in open source software, we shouldn't simply evaluate existing systems, but also explore possible variations and improvements to those systems.

Therefore, as I plan my research, I want to ensure that my work will support and advance usability in open source software. That's the reciprocal relationship.

I want to expand my previous research on GNOME usability. In a video chat a few weeks ago, GNOME designer Allan Day described GNOME's new design patterns, which they are working to integrate in a future GNOME version. I might explore these new design patterns—so my research question is probably along the lines of "How well do users navigate/understand/… the new design patterns in GNOME?" A practical approach to this research question would be to measure the success/impact of these design patterns on typical behavior/activities. And usability can be measured; it is a quantitative practice. One analytical method is to generate a "heat map" that demonstrates the relative ease or difficulty in accomplishing certain scenarios or actions.

Day suggested a list of GNOME applications that use the new design patterns concept. I might use any of these in my usability test:

  • Clocks
  • Contacts
  • Documents
  • Gedit (development version only)
  • Maps
  • Music
  • Notes
  • Photos
  • Software
  • Web

One way to explore the research question would be to conduct a usability test against prototypes (functional or on-paper) of the updated applications, using the new design patterns. By testing several applications that use the same or similar design patterns, the results might identify areas where the design patterns work well vs where they do not work well. These results would inform future GNOME design and development (reciprocal relationship).

What this blog is about

I just wanted to drop a note for anyone who's recently discovered my blog, to explain what this blog is about.

In short, I'm using this blog to record ideas and research on usability in open source software.

I'm an M.S. candidate in Scientific & Technical Communication, at the University of Minnesota. In September 2012, I began a directed study on usability. This was an excellent opportunity to tie my M.S. work to my outside interests in open source software. (I've written or contributed to lots of open source projects since about 1993, but I'm probably best known for starting the FreeDOS Project.) During my directed study, I needed a place to keep track of research, articles, and other things related to my directed study. I decided to start a blog.

When my directed study ended, I kept the blog "warm" by posting occasional thoughts and links to articles. It was a good exercise in my academic study, and a good place to keep notes—especially since I planned to use "open source software & usability" as the basis for my M.S. capstone project (similar to a thesis). So if you've been a longtime reader of my blog, you've probably noticed that posts started to slow down after January 2013.

But a year later, my M.S. capstone is starting to take off.

I plan to use this blog as a launching point for my M.S. capstone work. Again, I'll link to research and share a few thoughts about those articles. As my project swings into gear in the coming weeks, expect me to share more posts about usability.

If I could change one thing

I've been using GNOME 3.10 for over a month, since Fedora 20 came out. I really like it. But I've recently started thinking "If I could change one thing about GNOME to give it better usability, what would it be?" It's a tough mental exercise, to determine what one aspect I might change to make GNOME work even better for me.

I've previously mentioned the GNOME "hot corner" issue that came up in usability tests. I'll admit that I still run up against this problem. The "Activities" menu looks like a clickable item to me (and it is, if you don't activate the "hot corner") so when I try to use the menu, my natural desire is to click it. But just as I go to click it, my mouse cursor activates the "hot corner." So when I click (a split second later) the menu closes.

This is irritating, to be sure, but it's not the one thing I would change to improve GNOME. It's something I can work around. Changing that one little thing won't significantly improve GNOME's usability.

Instead, I would relocate the "application icon" menu into the window it's associated with.

My issue with the "application icon" menu is that it's disassociated with the window it controls. It's not too bad on a laptop or any system with a single display; in that case, the "application icon" menu isn't too far from the application's window. But at work, I use a dual-display setup; my laptop is the first display, and my 22-inch external monitor is my second display. I do all my "small screen real estate" functions on the laptop screen, including Rhythmbox music, the terminal, and the file manager. And I do all my "big screen real estate" functions on the big screen, such as word processing, spreadsheets, email, and web browsing. The "application icon" menu icon loses its association with the application window when they are separated on different displays. As a result, I often forget that menu functions may be located in the "application icon" menu.

To improve this, I would move the "application icon" menu out of the top bar, and into the application window. Here's a simple mock-up:

Mock-up of gedit, Nautilus, and Terminal; based on GNOME 3.10.3

I'm no graphics designer, but I've done my best to represent visually what I'm trying to describe. In this mock-up, GNOME's top bar is reserved for GNOME interaction: the "Activities" menu, the date & time, and the wifi & volume controls. The "application icon" menu gets moved into the active window. This provides a stronger relationship between the "application icon" menu and the application itself.

Actually, I cheated; I changed two things. In moving the "application icon" menu into the window, I also added a slight highlight to the active window. But that's a minor change.

Edit: I just realized that when I created my mock-up, I accidentally removed the "×" in the upper-right, to close the window. That's an error.

In this mock-up, only the active window has the "application icon" menu. This is intentional. That's one other way that GNOME would highlight the active window. And by having the "application icon" menu only appear on the active window (and disappear on inactive windows) the user would be more likely to notice the "application icon" menu, and thus draw a connection between the "application icon" menu and the application window.

Importantly, the "application icon" menu retains the downwards-triangle, which is a hint to the user that the menu is clickable. It's revealed to be a second menu. And menus are important; if the user cannot find the function he or she needs in the regular menu, they would now be more enticed to try to "application icon" menu.

Saturday, February 1, 2014

Users prefer familiar desktops

A recent article at Datamation discusses the paradox in KDE and GNOME: Polls show that KDE is the most popular Linux desktop, yet more users prefer desktops that use GNOME technology. What's the explanation?

From the article:

A likelier alternative is that the paradox has been misidentified. Perhaps the paradox is not technical in nature so much as an unintended consequence of the user revolts against KDE and GNOME and how they were handled.
From a user's perspective, KDE responded most effectively to its user revolt. Doing so was easy, because many of the features that users demanded to have back were already planned to be included in KDE 4.1. However, KDE also responded in a few months, with every sign of interest in readers' concerns.

However, GNOME's handling of its user revolt was very different. The general perception—which, needless to say, is not shared by GNOME's supporters—is that the project expected users to conform to the desktop design imposed by developers. And that it made no concession to complaints for almost eighteen months, when the official acceptance of GNOME extensions finally gave users the flexibility they were demanding.
More interestingly, the article also covers the user preference of a classic desktop style. Citing Linux Journal's 2013 Readers' Choice Awards, the authors note that "desktops that support a classic functionality, such as Xfce, MATE, and Cinnamon, were preferred by 61%." The article concludes with "For better or worse, they [users] know what they want—a classic desktop—and the figures consistently show that is what they are choosing in far greater numbers than GNOME, KDE, or any other single graphical interface."

These results are not that surprising. From my own usability test results, I found four themes of usability:

  1. Familiarity
  2. Consistency
  3. Menus
  4. Obviousness

Testers indicated that the programs seemed to operate more or less like their counterparts in Windows, and remember that most (6 out of 7) testers used Windows as their primary desktop. Gedit wasn't too different from Windows Notepad, or even Word. Firefox is like other browsers. Nautilus is very similar to Windows Explorer. To some extent, these testers had been trained under Windows, so having functionality—and paths to that functionality—that was approximately equivalent to the Windows experience was an important part of their success.

But it's good to see the same conclusion in the Datamation article.