Saturday, July 26, 2014

Interview for GNOME

I wanted to share this brief interview with me, for GUADEC and GNOME. You should also be able to find this on the GUADEC website in the coming days.

1. Nowadays many designers are interested in user experience rather than usability. Do you believe that usability alone is still relevant?
Usability and user experience are related, but different. Usability is about getting something done; user experience is about the user's emotional impression. Lots of things can affect the emotional experience of a graphical desktop like GNOME. Colors, fonts, location of elements, and window decorations are just some of the things that can influence how a person feels about using GNOME. That's the user experience.

Usability focuses on the user. The general rule about usability is that people use programs to be productive, and they are busy people who are trying to get things done. Through usability testing, the user decides when a product is easy to use. Because if a program is hard to use, no one will want to use it. And if they don't use the software, then they won't have an emotional experience about it.

So I believe that usability and user experience go hand-in-hand. Programs need to be pleasant (user experience) but people need to be able to use them, too (usability).
2. In your experience, what are the biggest difficulties you can incur in arranging an user testing?
It is critical to plan a usability test around the users. Who are the users? Do you only expect programmers to use it, or is it intended for a general audience? With GNOME, that means everyone, so any usability test of GNOME must be designed for "general users with average knowledge."

The next step is to decide what tasks those users need to do in GNOME. What are these general users trying to get done? In this usability test, we wanted to focus on new design patterns, but we first had to work out a set of tasks that real people would probably do: manage some folders and files, browse the web, take some notes, and so on.

Once you figure out what the usability test should cover, the hardest part is to make sure the tasks are realistic. You want each task to be something a real person would probably do in GNOME. But avoid using words or terms that actually appear in the program. That would only test if the user can match your task description to a menu item. Instead, you want to describe things using general terms. For example: when I asked testers to increase the font size on a website, I didn't use the word "font." Instead, the task was:

"You don't have your glasses with you, so it's hard to read the text on the website. Please make the text bigger on the website."
3. You have worked very much in last months in improving usability in GNOME: what was the hardest issue you found and the biggest satisfaction you have got?
I was really glad to see Allan and the other GNOME folks create entries in the GNOME Bugzilla. It's really satisfying to see GNOME developers taking usability seriously.

In doing the usability test, it is hard to watch someone struggle to complete a task. You can't give hints; you almost have to sit on your hands to keep from saying "the menu item you're looking for is right there." You must let the tester explore for themselves, in order to understand how users interact with your program.

I was surprised by some of the test results. For example: installing a program using Software. When testers searched for the program ("Robots") they got a list of programs that matched the search, and a convenient "Install" button they could click. But if they navigated through the categories to find the program, the "Install" button was in the upper-right corner, and users didn't see it. Instead, they clicked on the link to visit the program's website, which got them totally off track. So testers either completed this task very easily, or they were not able to do it at all.
4. What do you expect from this GUADEC?
This is my first time at GUADEC, so I really don't know what to expect. I have attended other similar conferences, so I expect to meet lots of interesting people. I am a very friendly person, so if you see me, please do say hi.

While I've visited other countries, this is my first trip to France. Unfortunately, I don't have any French, so I am hoping someone will help help keep me from getting lost. I also speak conversational Spanish and a little bit of Klingon, but neither will help me in France.
5. What can we expect from your keynote at GUADEC?
My keynote will be a summary of my usability research with GNOME. This is based on my Master's capstone project, which you can download from my blog: "Usability Themes in Open Source Software."

The presentation walks through the usability test of GNOME. I think folks will be very interested in the "heat map" of the usability test, which shows how testers fared in the test. It's a new way to share results of a usability test, and I think it helps to make issues more clear.

I will wrap up with a discussion of five themes from this usability test: how GNOME developers can extend this usability test to help them with other GNOME programs.

Thanks, and I look forward to meeting you at GUADEC!

Monday, July 21, 2014

Looking ahead to GUADEC

The 2014 GNOME Users And Developers European Conference ("GUADEC") starts this weekend in Strasbourg, France and I'm very excited to be there! This is my first experience at GUADEC. I will be speaking as a keynote, discussing my usability study of GNOME.

I am looking forward to so many of the presentations. Not all of them are strictly about GNOME, but also about free software and technology in general. Just to name a few:

As you can guess, several of these relate to my specific interest in the usability of free software.

I hope I see you there!

Thursday, July 10, 2014

Useful links and references

For those who are interested in joining the usability effort, I would like to share some useful links and references:

My research:
Jim Hall. "Usability Themes in Open Source Software." (EPUB and PDF)
GNOME : Usability Project » About : Usability methods

Wikipedia : Usability

STC SIG : Types of usability methods

Google : Types of usability (search query)
Janice (Ginny) Redish and Carol Barnum. "Overlap, Influence, Intertwining: The Interplay of UX and Technical Communication." Journal of Usability Studies, Volume 6, Issue 3, May 2011, pp. 90 - 101

David M Nichols, Michael B Twidale. "Usability Processes in Open Source Projects."

Calum Benson, Matthias Müller-Prove, Jiri Mzourek. "Professional Usability in Open Source Projects: GNOME,, NetBeans." » ACM

Nancy Frishberg , Anna Marie Dirks , Calum Benson , Seth Nickell , Suzanna Smith. "Getting to know you: open source development meets usability." CHI '02 Extended Abstracts on Human Factors in Computing Systems, April 20-25, 2002, Minneapolis, Minnesota, USA

Michael J. Karels, Commercializing Open Source Software, Queue, v.1 n.5, July/August 2003

Monday, June 16, 2014

Usability in cars

When you create a new system, the first step isn't how to design the program's codebase, or what functions and I/O calls you will need to make. The firs step should be to ask "what will the user need to do with this system?" That question drives the design decisions of how the user will interact with your program, and how your system will provide feedback to the user.

Unfortunately, there are too many examples where system designers choose to shove more functionality into the application, because they can. But when you have too much information, or when the information is too dense or obtuse for an average user with typical knowledge to understand, your design becomes an impediment to understanding. Perhaps a beautiful impediment to understanding, but an impediment nonetheless.

And so it is with all applications and interfaces. While my blog focuses on open source software and free software, poor usability is not a phenomenon limited to free software. Proprietary systems have this problem, too.

For example, car in-dash systems. Most modern cars today have some variant of an interactive dashboard; it seems the venerable knob or the reliable switch is about to disappear. In-dash systems provide a wonderful opportunity for car designers; they can add control interfaces for all kinds of things: entertainment, environment, navigation, and other car systems.

But it turns out, drivers don't want that. According to a recent usability study by automotive market research firm SBD, drivers only want to access music and navigation via the in-dash interface. Anything else is just visual noise, clutter that obscures the critical functionality that drivers are looking for. Look at it this way: if you're driving down the freeway at 60 MPH, do you want to spend the next 30 seconds or a minute exploring menus of extraneous functionality just to find the feature you want? A lot can happen in a minute when you're traveling at that speed. That's a mile for a deer to walk across the road, for another driver to change lanes, for traffic to back up. And all you wanted was to select a radio station on satellite.

Personally, I've always believed that big, obvious icons are important for an in-dash system. They are easy to understand at a glance, and their larger size makes it easier to jab at an icon when you'd rather keep your full attention on the road. So I was pleased to see Tesla's larger icons fare better in the usability study:

"We were very enamored with the Tesla. Not being tech -savvy, I found the icons were huge and easy to use while driving and while parked," said Tina, another focus group member. "It was a fascinating system to me, and I'd seek that one out [as a car buyer]."

But perhaps my favorite quote from the article spoke to the issue of building too much complexity into what should be a simple system, for the sake of adding new features: "As an industry, we're striving to develop Swiss Army knives instead of the spoons our customers want," Hart [Andrew Hart, head of advanced research at SBD] said. Don't build a Swiss Army knife when you just need a spoon. Build a great spoon instead.
photo: Kyle May

Sunday, June 15, 2014

An introduction to Linux

News about this has been out for some time, but in case you hadn't heard, I wanted to direct your attention to this item: the Linux Foundation announced in March that it would be working with edX, a non-profit online learning site governed by Harvard and MIT, to make its “Introduction to Linux” course free and open to all.

While this isn't specifically related to Open Source Software and Usability, I think it's great that edX and LF will help make Linux training available to everyone. From the edX site: "Upon completion of this training you should have a good working knowledge of Linux, from both a graphical and command line perspective, allowing you to easily navigate through any of the major Linux distributions. You will be able to continue your progress as either a user, system administrator or developer using the acquired skill set."

Classes start in August (2014). edX expects that students would spend anywhere from 40 to 60 hours to cover the course content.
photo: Larry Ewing

Friday, May 16, 2014

Free software outpaces proprietary code quality

I was interested to read this recent article in CIOL, describing the analysis of 750 million lines of free software and open source software code through the Coverity Scan service, and comparing commercial usage of the Coverity Development Testing Platform. According to the article, this is the largest sample size that the report has studied to date. Their results: free software code quality outpaces proprietary code code quality.

Key findings from the report include:

  • Open source code quality surpasses proprietary code quality in C/C++ projects. Coverity's analysis found an average defect density of .59 for open source C/C++ projects that leverage the Scan service, compared to an average defect density of .72 for proprietary C/C++ code developed for enterprise projects.
  • Linux continues to be a benchmark for open source quality.
  • C/C++ developers fixed more high-impact defects. Not so much for projects written in Java, for example.

While I would be cautious about overconfidence, these results support Eric S. Raymond's maxim (which he terms "Linus's Law") that "given enough eyeballs, all bugs are shallow," from The Cathedral and the Bazaar.

Saturday, May 10, 2014

Usability testing through prototypes

Usability in all software is a problem, not just free software and open source software. But with free software and open source software, we have a unique opportunity. Our users are our collaborators. We often think of this from a "developer" mindset; it really helps to have people out there writing code, submitting bug fixes, adding new features. Sometimes we step outside this mode of thinking and ask collaborators to do other helpful things such as writing documentation, testing functionality, and reporting bugs. With everyone working together, we all win.

But in free software and open source software, our user/collaborators can and should include usability testing. Developers can learn a lot about how their programs get used just by sitting down with a few people and watching them explore the program. You can use a variety of methods to do usability testing; my paper lists 11 methods, and demonstrates a full usability test. But there are other ways to do usability testing.

One way is a variation of the "Paper prototype" method. In a traditional paper prototype test, you print out on paper all the screens that the user might encounter when using the program. Then you ask people to simulate using the program by telling you what they might click on, and you present new menus or screens for them to interact with.

But it's hard to do a paper prototype test when you're a small free software or open source software project. Getting people together to test can take some doing.

Instead, leverage a website to do the prototype testing for you. Create a few mockups of your program screens, showing every menu that users can interact with. Then "animate" these using a series of simple web pages. The tricky part is getting all the screens in order. That means you'll need to grab screenshots of your program, or create new screenshot mockups.

If your program includes the menus and functionality already, you can just grab screenshots for each menu and every screen. If your program isn't quite "there" yet, then you'll need to creat screenshot mockups to use. Here's an example:

Let's say I wanted to write a simple Tic Tac Toe game program. I'm not sure what menus I'll use, or what items I would need to use in each menu. So I decide to create screenshot mockups. To start, I'll grab a screenshot of another program, just to get all the major screen elements in order. Here's the GNOME Terminal program:

(click to enlarge)

Open this screenshot image into GIMP, and do some quick edits to remove the "Terminal" parts of the image. This is the first step to creating simple screenshot mockups.

(click to enlarge)

Add elements that represent what your program screen might look like. In this example, I've added the tic tac toe grid, edited the title, and simplified the menus to suit my game:

(click to enlarge)

Note that the screenshot mockup doesn't need to look exactly like the final version of the program. This is just a mockup that testers can recognize. Don't spend a lot of time making this perfect. You can see in my example that I wasn't too careful in removing all the elements from the original Terminal screenshot, and I created a rough tic tac toe grid by drawing two rectangles on the screen and chopping off the ends. It's not perfect, but testers will be able to understand the mockup if it is at least close. You can always add a disclaimer later that the mockup is just that: a simulation of what the program screens might look like.

The final step is to create screens showing all the menus and other interactions that your program might allow. In this example, I only have a few menus. Maybe your program also allows right-click context sensitive menus, or uses pop-up windows to display other information. You'll need to create screenshot mockups for these screens as well. Here's one screenshot for the File menu in my Tic Tac Toe game. It's not perfect, but it's enough for people to understand and interact with the screenshot:

(click to enlarge)

Once you create the menus for your program, create a very simple website that lets testers explore the mockup. No need to use fancy javascript here; you can do it very simply. For example, you might put each screenshot mockup on its own web page, and use the html <area> tag to to define regions where users can click to view the next screenshot mockup, on a different web page. In my Tic Tac Toe game, I might use <area shape="rect"> for each of the red rectangles, to let testers access each of the menus in my game program. So testers can "exit" out of the menus, I might define <area shape="poly"> for the purple region:

(click to enlarge)

I'll add that it took me all of 30 minutes to create these five simple screenshot mockups in GIMP. This doesn't need to take a lot of time. To create mockups for the other menus might take me another 20 minutes.

Having created the screenshot mockups and the simple website that lets users explore the mockup, you'll need to invite testers to try out the mockup. Define a few simple scenarios that "exercise" different parts of the program. I find it's best to limit testers to two or three scenarios. Create a separate web page with these scenarios, and ask testers to open the "mockup website" in a separate tab in their web browser. They can flip back and forth between the scenarios and the mockup as they test.

Your scenarios should be worded very simply and plainly, but avoid using terms from the program. For example, for the Tic Tac Toe game, you might ask the tester to change the "look" of the tic tac toe board, not change its "view."

Testers can then run through the mockup and click on each screen as they test the scenario. At the end of each scenario, ask the testers to fill out a brief survey, such as via a simple web form:
  1. Do you think you completed this task?
  2. Please describe what you did (what you clicked on) to complete the task.
  3. What worked well?
  4. What would you change?
  5. Rate the difficulty of this task. (You might use a form with a numerical scale, -5 to +5, where -5 is hard, and +5 is easy.)
You can learn a lot about your program if you ask just a few users to run through these screenshot mockups. What you think is easy might be challenging for someone else. Menu items or descriptions that make sense to you might be difficult for someone else to understand. But by running through a simple "animated prototype" described here, you can quickly find out what areas of your program need more work, and improve the usability. And with good usability, everyone wins.