Monday, July 21, 2014

Looking ahead to GUADEC

The 2014 GNOME Users And Developers European Conference ("GUADEC") starts this weekend in Strasbourg, France and I'm very excited to be there! This is my first experience at GUADEC. I will be speaking as a keynote, discussing my usability study of GNOME.

I am looking forward to so many of the presentations. Not all of them are strictly about GNOME, but also about free software and technology in general. Just to name a few:

As you can guess, several of these relate to my specific interest in the usability of free software.

I hope I see you there!

Thursday, July 10, 2014

Useful links and references

For those who are interested in joining the usability effort, I would like to share some useful links and references:

My research:
Jim Hall. "Usability Themes in Open Source Software." (EPUB and PDF)
GNOME : Usability Project » About : Usability methods

Wikipedia : Usability

STC SIG : Types of usability methods

Google : Types of usability (search query)
Janice (Ginny) Redish and Carol Barnum. "Overlap, Influence, Intertwining: The Interplay of UX and Technical Communication." Journal of Usability Studies, Volume 6, Issue 3, May 2011, pp. 90 - 101

David M Nichols, Michael B Twidale. "Usability Processes in Open Source Projects."

Calum Benson, Matthias Müller-Prove, Jiri Mzourek. "Professional Usability in Open Source Projects: GNOME,, NetBeans." » ACM

Nancy Frishberg , Anna Marie Dirks , Calum Benson , Seth Nickell , Suzanna Smith. "Getting to know you: open source development meets usability." CHI '02 Extended Abstracts on Human Factors in Computing Systems, April 20-25, 2002, Minneapolis, Minnesota, USA

Michael J. Karels, Commercializing Open Source Software, Queue, v.1 n.5, July/August 2003

Monday, June 16, 2014

Usability in cars

When you create a new system, the first step isn't how to design the program's codebase, or what functions and I/O calls you will need to make. The firs step should be to ask "what will the user need to do with this system?" That question drives the design decisions of how the user will interact with your program, and how your system will provide feedback to the user.

Unfortunately, there are too many examples where system designers choose to shove more functionality into the application, because they can. But when you have too much information, or when the information is too dense or obtuse for an average user with typical knowledge to understand, your design becomes an impediment to understanding. Perhaps a beautiful impediment to understanding, but an impediment nonetheless.

And so it is with all applications and interfaces. While my blog focuses on open source software and free software, poor usability is not a phenomenon limited to free software. Proprietary systems have this problem, too.

For example, car in-dash systems. Most modern cars today have some variant of an interactive dashboard; it seems the venerable knob or the reliable switch is about to disappear. In-dash systems provide a wonderful opportunity for car designers; they can add control interfaces for all kinds of things: entertainment, environment, navigation, and other car systems.

But it turns out, drivers don't want that. According to a recent usability study by automotive market research firm SBD, drivers only want to access music and navigation via the in-dash interface. Anything else is just visual noise, clutter that obscures the critical functionality that drivers are looking for. Look at it this way: if you're driving down the freeway at 60 MPH, do you want to spend the next 30 seconds or a minute exploring menus of extraneous functionality just to find the feature you want? A lot can happen in a minute when you're traveling at that speed. That's a mile for a deer to walk across the road, for another driver to change lanes, for traffic to back up. And all you wanted was to select a radio station on satellite.

Personally, I've always believed that big, obvious icons are important for an in-dash system. They are easy to understand at a glance, and their larger size makes it easier to jab at an icon when you'd rather keep your full attention on the road. So I was pleased to see Tesla's larger icons fare better in the usability study:

"We were very enamored with the Tesla. Not being tech -savvy, I found the icons were huge and easy to use while driving and while parked," said Tina, another focus group member. "It was a fascinating system to me, and I'd seek that one out [as a car buyer]."

But perhaps my favorite quote from the article spoke to the issue of building too much complexity into what should be a simple system, for the sake of adding new features: "As an industry, we're striving to develop Swiss Army knives instead of the spoons our customers want," Hart [Andrew Hart, head of advanced research at SBD] said. Don't build a Swiss Army knife when you just need a spoon. Build a great spoon instead.
photo: Kyle May

Sunday, June 15, 2014

An introduction to Linux

News about this has been out for some time, but in case you hadn't heard, I wanted to direct your attention to this item: the Linux Foundation announced in March that it would be working with edX, a non-profit online learning site governed by Harvard and MIT, to make its “Introduction to Linux” course free and open to all.

While this isn't specifically related to Open Source Software and Usability, I think it's great that edX and LF will help make Linux training available to everyone. From the edX site: "Upon completion of this training you should have a good working knowledge of Linux, from both a graphical and command line perspective, allowing you to easily navigate through any of the major Linux distributions. You will be able to continue your progress as either a user, system administrator or developer using the acquired skill set."

Classes start in August (2014). edX expects that students would spend anywhere from 40 to 60 hours to cover the course content.
photo: Larry Ewing

Friday, May 16, 2014

Free software outpaces proprietary code quality

I was interested to read this recent article in CIOL, describing the analysis of 750 million lines of free software and open source software code through the Coverity Scan service, and comparing commercial usage of the Coverity Development Testing Platform. According to the article, this is the largest sample size that the report has studied to date. Their results: free software code quality outpaces proprietary code code quality.

Key findings from the report include:

  • Open source code quality surpasses proprietary code quality in C/C++ projects. Coverity's analysis found an average defect density of .59 for open source C/C++ projects that leverage the Scan service, compared to an average defect density of .72 for proprietary C/C++ code developed for enterprise projects.
  • Linux continues to be a benchmark for open source quality.
  • C/C++ developers fixed more high-impact defects. Not so much for projects written in Java, for example.

While I would be cautious about overconfidence, these results support Eric S. Raymond's maxim (which he terms "Linus's Law") that "given enough eyeballs, all bugs are shallow," from The Cathedral and the Bazaar.

Saturday, May 10, 2014

Usability testing through prototypes

Usability in all software is a problem, not just free software and open source software. But with free software and open source software, we have a unique opportunity. Our users are our collaborators. We often think of this from a "developer" mindset; it really helps to have people out there writing code, submitting bug fixes, adding new features. Sometimes we step outside this mode of thinking and ask collaborators to do other helpful things such as writing documentation, testing functionality, and reporting bugs. With everyone working together, we all win.

But in free software and open source software, our user/collaborators can and should include usability testing. Developers can learn a lot about how their programs get used just by sitting down with a few people and watching them explore the program. You can use a variety of methods to do usability testing; my paper lists 11 methods, and demonstrates a full usability test. But there are other ways to do usability testing.

One way is a variation of the "Paper prototype" method. In a traditional paper prototype test, you print out on paper all the screens that the user might encounter when using the program. Then you ask people to simulate using the program by telling you what they might click on, and you present new menus or screens for them to interact with.

But it's hard to do a paper prototype test when you're a small free software or open source software project. Getting people together to test can take some doing.

Instead, leverage a website to do the prototype testing for you. Create a few mockups of your program screens, showing every menu that users can interact with. Then "animate" these using a series of simple web pages. The tricky part is getting all the screens in order. That means you'll need to grab screenshots of your program, or create new screenshot mockups.

If your program includes the menus and functionality already, you can just grab screenshots for each menu and every screen. If your program isn't quite "there" yet, then you'll need to creat screenshot mockups to use. Here's an example:

Let's say I wanted to write a simple Tic Tac Toe game program. I'm not sure what menus I'll use, or what items I would need to use in each menu. So I decide to create screenshot mockups. To start, I'll grab a screenshot of another program, just to get all the major screen elements in order. Here's the GNOME Terminal program:

(click to enlarge)

Open this screenshot image into GIMP, and do some quick edits to remove the "Terminal" parts of the image. This is the first step to creating simple screenshot mockups.

(click to enlarge)

Add elements that represent what your program screen might look like. In this example, I've added the tic tac toe grid, edited the title, and simplified the menus to suit my game:

(click to enlarge)

Note that the screenshot mockup doesn't need to look exactly like the final version of the program. This is just a mockup that testers can recognize. Don't spend a lot of time making this perfect. You can see in my example that I wasn't too careful in removing all the elements from the original Terminal screenshot, and I created a rough tic tac toe grid by drawing two rectangles on the screen and chopping off the ends. It's not perfect, but testers will be able to understand the mockup if it is at least close. You can always add a disclaimer later that the mockup is just that: a simulation of what the program screens might look like.

The final step is to create screens showing all the menus and other interactions that your program might allow. In this example, I only have a few menus. Maybe your program also allows right-click context sensitive menus, or uses pop-up windows to display other information. You'll need to create screenshot mockups for these screens as well. Here's one screenshot for the File menu in my Tic Tac Toe game. It's not perfect, but it's enough for people to understand and interact with the screenshot:

(click to enlarge)

Once you create the menus for your program, create a very simple website that lets testers explore the mockup. No need to use fancy javascript here; you can do it very simply. For example, you might put each screenshot mockup on its own web page, and use the html <area> tag to to define regions where users can click to view the next screenshot mockup, on a different web page. In my Tic Tac Toe game, I might use <area shape="rect"> for each of the red rectangles, to let testers access each of the menus in my game program. So testers can "exit" out of the menus, I might define <area shape="poly"> for the purple region:

(click to enlarge)

I'll add that it took me all of 30 minutes to create these five simple screenshot mockups in GIMP. This doesn't need to take a lot of time. To create mockups for the other menus might take me another 20 minutes.

Having created the screenshot mockups and the simple website that lets users explore the mockup, you'll need to invite testers to try out the mockup. Define a few simple scenarios that "exercise" different parts of the program. I find it's best to limit testers to two or three scenarios. Create a separate web page with these scenarios, and ask testers to open the "mockup website" in a separate tab in their web browser. They can flip back and forth between the scenarios and the mockup as they test.

Your scenarios should be worded very simply and plainly, but avoid using terms from the program. For example, for the Tic Tac Toe game, you might ask the tester to change the "look" of the tic tac toe board, not change its "view."

Testers can then run through the mockup and click on each screen as they test the scenario. At the end of each scenario, ask the testers to fill out a brief survey, such as via a simple web form:
  1. Do you think you completed this task?
  2. Please describe what you did (what you clicked on) to complete the task.
  3. What worked well?
  4. What would you change?
  5. Rate the difficulty of this task. (You might use a form with a numerical scale, -5 to +5, where -5 is hard, and +5 is easy.)
You can learn a lot about your program if you ask just a few users to run through these screenshot mockups. What you think is easy might be challenging for someone else. Menu items or descriptions that make sense to you might be difficult for someone else to understand. But by running through a simple "animated prototype" described here, you can quickly find out what areas of your program need more work, and improve the usability. And with good usability, everyone wins.

Friday, May 9, 2014

What's next?

As you know from my other post, my Master's capstone title was Usability Themes in Open Source Software. I feel very passionate about this topic, and I plan to continue working with usability in free software and open source software. As a start, I will keep writing here.

I'm going to submit a version of my capstone project to the Journal of Usability Studies. I think I can get three good JUS articles out of it: the usability study & results, using a heat map to display usability results, and a lit review of usability in open source software.

Both Linux Journal and Linux Voice said they would like to run a version of my article, so I'll write separate versions for them. And the blog has asked me to write an ongoing series of articles about usability in open source software. And I will be submitting proposals at open source software conferences like GUADEC, O'Reilly OSCON, Penguicon, and FOSDEM. I hope to see you there!

And the GNOME folks have asked if I can help them with future usability tests of new versions of GNOME. That would be a great continuation of this work!
Subscribe to the RSS feed to stay updated on Open Source Software & Usability! (RSS)