Friday, December 21, 2018

Top ten of 2018

As always, I enjoy writing about the usability of open source software. This year, I've written several articles about usability—some on my blog, others as articles in journals and websites. As I look back on the last twelve months, I thought it would be interesting to highlight a few of my favorite blog posts from the year. And here they are, in no particular order:

1. Why the Linux console has sixteen colors
At the 2018 Seattle GNU/Linux Conference after-party, I gave a lightning talk about why the Linux console has only sixteen colors. Lightning talks are short, fun topics. I enjoyed giving the lightning talk, and the audience seemed into it, too. So I thought I'd share my lightning talk here. These are my slides in PNG format, with notes added.
2. Usability Testing in Open Source Software
I recently attended the 2018 Seattle GNU/Linux Conference, where I gave a presentation about usability testing in open source software. I promised to share my presentation deck. Here are my slides in PNG format, with notes added.
3. Making a first contribution in Outreachy usability testing
This year, I decided to get back into the Outreachy program as a mentor. Outreachy is a paid internship program for women and others who are under-represented in open source software. As part of an intern's application process, we ask that they make a first contribution to their project. Since I would mentor usability testing for GNOME, this post explained how to do a usability test as part of that initial contribution. I think it's a good starting point for anyone who wants to get into usability testing.
4. Open source tools I used to write my latest book
I recently wrote a book about FreeDOS. Using FreeDOS is my celebration of the 24th anniversary of FreeDOS. This is a collection of how-tos about installing and using FreeDOS, essays about my favorite DOS applications, and quick-reference guides to the DOS command line and DOS batch programming. I've been working on this book over several months, with the help of a great professional editor. The book was produced almost entirely with open source software. I'd like to share a brief insight into the tools I used to create, edit, and produce Using FreeDOS.
5. The next step in open data is open source
For the last three years, I have served as CIO in local government. Governments at all levels are moving to embrace open data, where governments share public data proactively with citizens. Open data can be used, reused, mixed, and shared by anyone. I believe that next step is open source. Where we provide government data sets for anyone to view, adapt, and remix, we need to offer government source code for others to view, adapt, and remix.
6. What an icon says about you
When I first started doing usability testing, I focused on icons. How will users perceive the function exposed by a single button, especially if that button is an icon? Icons can be abstract or representative. For example: in a word processing program, what icon would you click to print a document? In this essay, I discuss elements of icons, and offer advice to Firefox as they proposed a new icon set.
7. A few thoughts on GNOME usability
I recently learned that GNOME developers have proposed moving the application menu off the "top black bar" and into the application window. In this post, I shared a few thoughts on the new design concept.
8. A few thoughts on copyright
Every once in a while, I'll come across a discussion where someone justifies pirating a movie or popular TV show with "nothing of value was lost." Basically, these people claim that it isn't really "stealing" if the content creator (HBO, Disney, etc) keeps the original copy. But whether you realize it or not, copyright protection works for more than just the Big Media companies (HBO, Disney, etc). Copyright works for Free software and open source software, too. In fact, the copyleft afforded by the GNU General Public License only works because of copyright protections.
9. Open source software reading list
A colleague had asked what books I'd recommend about open source software. I go back a ways with open source software. I first contributed to Free software and open source software in 1993, before the term "open source software" was widely adopted. So my list of book recommendations has some older titles on there. And that's good, because this list also provides a solid grounding for contributing to open source software.
10. What open source software programs I love
Earlier this year, someone asked me what Free software and open source software programs I really love. I thought I'd share that here, too. As I started to go through my favorite programs, I realized it was quite long. So I'm trying to keep the list short here, just the programs I use the most.

Saturday, December 8, 2018

Week 1 of GNOME usability testing

The Outreachy internship started this week! For this cycle, we are joined by Clarissa, who will help us with usability testing in GNOME.

I wanted to share our progress in the internship. I hope to provide regular status updates on our work.

For this week, I want to start with a grounding in Usability testing. You can do usability testing in different ways. Alice Preston described the types of usability methods for the STC newsletter (Vol 10, No. 3). I can't find a copy of that newsletter online, but in the list, she describes 11 ways to evaluate usability in a design. Preston's list is:

  1. Interviews/Observations: One-on-one sessions with users.
  2. Focus Groups: Often used in marketing well before there is any kind of prototype or product to test, a facilitated meeting with multiple attendees from the target user group.
  3. Group Review or Walk-Through: A facilitator presents planned workflow to multiple attendees, who present comments on it.
  4. Heuristic Review: Using a predefined set of standards, a professional usability expert reviews someone else's product or product design and presents a marked checklist back to the designer.
  5. Walk-Around Review: Copies of the design/prototype/wireframe are tacked to the walls, and colleagues are invited to comment.
  6. Do-it-Yourself Walk-Through: Make mock-ups of artifacts, but make the scenarios realistic. Walk through the work yourself.
  7. Paper Prototype Test: Use realistic scenarios but a fake product.
  8. Prototype Test: A step up from a paper prototype, using some type of animated prototype with realistic scenarios.
  9. Formal Usability Test: Using a stable product, an animated prototype, or even a paper prototype, test a reasonably large number of subjects against a controlled variety of scenarios.
  10. Controlled Experiment: A comparison of two products, with careful statistical balancing, etc
  11. Questionnaires: Ask testers to complete a formal questionnaire, or matching questionnaire.


Most of our work in the internship will be testing designs that haven't gone "live" yet (this is called "prototype usability testing"). Allan and Jakub will create mock-ups of new designs, and Clarissa will do usability testing on them. You can read a bit about this on Allan's blog post.

In that post, Allan writes:
Therefore, for this round of the Outreachy internships, we are only going to test UX changes that are actively being worked on. Instead of testing finished features, the tests will be on two things:
  1. Mockups or prototypes of changes that we hope to implement soon (this can include static mockups and paper or software prototypes)
  2. Features or UI changes that are being actively worked on, but haven’t been released to users yet

Ciarrai interned with us a few years ago, and worked on prototype usability testing for the then-upcoming GNOME Settings app redesign. We wrote an article together for OpenSource.com about paper-based usability testing that you might find interesting.

You may also be interested in an older article I wrote about usability testing with prototypes. I don't expect that we'll be testing with the animated prototypes that my article proposes, but it's good background material.

As you prepare for usability testing, you may ask "How many testers do I need?" The answer is "about five" if you are doing a traditional usability test. For a paper prototype test, this may be different. How many do you think we need?

For additional background, you might be interested to read this essay about how many testers you need, and why the answer is "about five." Note that "five" only applies to doing usability testing in a traditional or "formal" usability test, where you test real people with real scenario tasks against a real product. For paper prototype testing, you have a different value for L, so we'll need a different number of testers. But this is a good start to understanding the assumptions.

Usability expert Jakob Nielsen has a good video about "five testers" that I often recommend.

Looking for more background information? Here are a few additional articles related to "how many testers do I need?"



That last article makes an interesting point that I'll quote here: "Studies to evaluate a prototype of a novel user-interface design often concern the discovery of severe show-stopper problems. Testing usually reveals such severe errors relatively quickly, so these tests often require fewer participants."

But, how many testers do you think you'll need for each iteration of prototype testing?