Wednesday, September 26, 2012

Usability processes in open source software

David Nichols and Michael Twidale wrote an excellent article entitled Usability Processes in Open Source Projects that is an excellent match for my study. Nichols, D.M. & Twidale, M.B. (2003). The Usability of Open Source Software. First Monday 8(1).

They summarize how usability is a problem in open source software in four succinct points:
  1. Developers are not typical end users
  2. Usability experts usually do not get involved in open source software
  3. Open source projects often lack usability resources
  4. Developers do not approach interface design in the same way they do functionality design
I find this to be very true. In my own experience, I find that most open source projects are written by developers for other developers. That is, the developers are not typical end users. This may be okay if the developer is creating a developer tool, something so technical that "average" end users are unlikely to use it. But for the bulk of popular open source software, the developers are not like the typical end users. And unfortunately, open source software projects do not tend to attract usability experts.

Nichols and Twidale dicuss the complexity of open source software usability, saying the existence of certain usability bugs may be viewed as subjective and unreliable by open source developers. I view this as a cultural issue in the open source software community. As Nichole and Twidale mention, usability bugs just aren't perceived as having the same weight as functionality bugs. If a feature is broken, it needs to get fixed. If a feature is hard to use, you probably haven't read the manual.

You see this all the time in open source software projects. Bugzilla (a popular open source bug-tracking tool) allows bugs to be marked "WORKSFORME," which indicates the bug cannot be replicated by a developer. And usability bugs such as "this feature is hard to use" technically do work so often get closed with "WORKSFORME."

Nichols and Twidale's comment that "discussions need to involve complex design rationales, multiple, prehaps contradictory goals and design trade-offs" may be another indicator of cultural issues in open source software development. Many open source developers (certainly the ones I know) approach their work in a logical, ordered fashion. Design issues that involve trade-offs may be dismissed, in preference to work on bigger functionality enhancements.

The bug-reporting method needs to be different for usability bugs versus functionality bugs. Nichols and Twidale suggest creating mock-ups in HTML, "ASCII art," drawing packages, sketches, and interface design toolkits. After all, "a picture is worth a thousand words" - it's easier to show the problem and propose a solution via a simple drawing than to describe it in words.

So how can open source software projects improve their usability? Nichols and Twidale suggest several options:
  1. Improved bug reporting, by lowering the cultural, technical, and usability barriers. A usability "bug" tracking tool should be separate from a system that tracks functional bugs.
  2. Improved analysis of usability issues. In a 2003 study, Nichols and Twidale adocate creating a method for very small scale usability studies - as short as 10 minutes for a single user. [I am very interested in their suggestion. This may be helpful in a later phase of my study.] However, open source projects may be challenged in achieving consensus when changing an interface.
  3. Redesign work, and moving design work elsewhere. For example, "design by blog" may be used to offload design ideas from functional programming efforts. For example, Evolution (an open source email and calendar system) discusses design ideas on a separate blog. By working through different design ideas, and sharing mock-ups proposed by users, the blog can build consensus for new interfaces in a transparent way.

Wednesday, September 19, 2012

GNOME,, NetBeans

I've been reviewing other studies of open source usability. Benson, Müller-Prove, and Mzourek shared a study entitled "Professional Usability in Open Source Projects: GNOME,, NetBeans." CHI EA '04 CHI '04 extended abstracts on Human factors in computing systems. pp 1083-1084. ACM New York, NY, USA. 2004.

They acknowledge that open source applications tend to "have poor user interfaces." This is because most open source programs "are created by engineers for engineers. The feedback cycle with real users does not exist because there are few usability experts participating in open source development processes." I've used a similar phrase: written by developers for other developers. I agree with their conclusion that the community feedback rarely includes useful usability information. In my experience, the UI experts are either not contributing, or developers discount UI expertise.

Benson, Müller-Prove, and Mzourek don't provide a deep analysis. They quickly summarize (in two pages) the state of three high-visibility open source projects: GNOME (a desktop environment, similar to the desktop on MacOSX or Microsoft Windows), (an office suite, including "workalikes" for Microsoft Word, Excel, and Powerpoint), and NetBeans (an integrated development environment for the Java programming language).

Here is a summary of their study:

GNOME - challenges:
  • Communication: Dedicated usability mailing lists and IRC channels do exist but they host few useful discussions and, along with the complex bug database, can be intimidating to non-technical contributors.
  • Early input: The GUP acts less as a usability “consultancy” for developers and more as a group of individuals who comment on usability bugs. Although the open source philosophy of contributing code early and often should lend itself well to rapid iterative prototyping, usability assistance is usually only sought after interfaces have been “designed”.
  • User profiles: There is no shared or documented vision of GNOME’S target audience.
  • Process: No obvious usability methodology is employed. Many developers contribute in their spare time for fun, so are understandably reluctant to follow heavyweight processes.
  • Attitude: Some developers feel that “Sun will fix that, so we don’t need to bother”. - information:
  • was open sourced in 2000, after Sun Microsystems purchased StarOffice, a commercial office suite.
  • The StarOffice User Experience Team is involved in the development of, and includes professionals from all fields of HCI , including GUI and interaction design, usability engineering, linguistic expertise, accessibility, and globalization.

NetBeans - challenges:
  • Communication: Discussion is fragmented between the mailing list and bug reports. These channels need a clearer charter and decision-making system.
  • User profiles: Are mailing list participants really our target audience? If not, who should we optimize for?
  • Responsibility: Who owns the functionality? The developer, who writes and owns the code, or the designer, who writes the specification? A clearer notion of shared ownership is required.

It is interesting to capture their findings, but otherwise I view it as a very general reference.

Tuesday, September 18, 2012

Deep dive

My goal in this study is to do a "deep dive" on the usability of open source software user interfaces, and examine how usability might be applied to open source software programs. But what do I mean by a "deep dive"?

I mean I intend to an in-depth exploration,¹ to review open source usability in extensive detail.²

This is a topic that is very interesting for me. I am actively engaged in free and open source software. I first discovered free software in 1992 while I was an undergrad. Of course, I'd used shareware (programs that you could try for free, but you were expected to pay if you continued to use them) on MS-DOS for years. While a student, I used As-Easy-As (a shareware spreadsheet program) and Galaxy Write (a shareware word processor) for working on papers and analyzing my lab data. Our computer labs used SunOS and VAX, and the administrator provided free software tools such as Emacs (a programmer's editor) and gcc (a free compiler) for us to use. I found these to be the same high quality as the shareware programs I'd used.

In 1993, I heard about a new operating system that could replace MS-DOS on my computer. This new system was called "Linux", and was built from free software (the term open source software hadn't been coined yet). All the tools we used on the SunOS systems were present in Linux, so I gave it a go. Suddenly, I had the same power of a SunOS workstation on my home computer. No more late-night trips to the computer lab!

But it wasn't just about getting a free ride. For me, the real benefit of free software was that you had access to the source code. If you had the interest and the skill, you could modify the programs to do exactly what you wanted. If you found a bug in a program, you could fix it. If you needed a new feature that would improve the program, you could do that yourself. And over time, I did make a few improvements here and there, and shared my changes with the people who maintained those programs. Some of my changes even made it into future versions of the programs.

Such was my interest in this new way of using software that I created my own free operating system. In 1994, Microsoft announced that their next version of Windows would completely replace MS-DOS. I still used MS-DOS to do a few things (I needed As-Easy-As to do my lab analysis, for example) and didn't want to see DOS go away. So I started work to create a free version of DOS, which later became FreeDOS.

I've continued to use free / open source ever since. Much of the time, I prefer to use Linux as my desktop. But even when I'm using Windows or MacOSX, I like to use Chrome and Firefox - both are open source software (Chrome is built on the open source browser, Chromium). I'm not exclusive, however; I'll use tools and programs as they suit my needs.

At the same time, I have long recognized that open source software lacks a certain UI polish. Often, these programs are written by developers for other developers. Or a programmer picks up a problem to "scratch an itch" (from Eric S. Raymond's The Cathedral and the Bazaar) and in doing so, focuses so intently on the functional problem that he/she often forgets about the interface.

So I want to investigate this problem at a deeper level, to apply what I understand about user interfaces and usability, to study the usability of open source software. I think the community or audience that would benefit from a usability study on open source software is the open source community itself.

Thursday, September 13, 2012

Thought leaders

I'm turning my thoughts towards the thought leaders in this space. There hasn't been much work in usability with open source software, but there has been some. I would start with the references who have discussed open source software usability:

  • Calum Benson (Calum also announced the first GNOME usability study in 2001)
  • Matthias Müller-Prove
  • Jiri Mzourek
  • David M Nichols
  • Michael B Twidale
I would add to this list other thought leaders in user experience and usability:
Since a target audience for my research is the open source community, I also plan to seek out several leaders in that community:
  • Shawn Powers (of Linux Journal)
  • Eric S. Raymond
Who else should I contact? If you have suggestions, please leave them in the comments.

Wednesday, September 12, 2012

Other references

Aside from the GNOME usability effort, I found two other references about usability in open source software projects:

Calum Benson, Matthias Müller-Prove, Jiri Mzourek. "Professional Usability in Open Source Projects: GNOME,, NetBeans." Proceeding, CHI EA '04, CHI '04 extended abstracts on Human factors in computing systems, pp. 1083-1084, ACM New York, NY, USA 2004.
(available from ACM)
Working as a usability professional in the open source arena is a challenging task. The decentralized and engineering-driven approach of open source projects can be at odds with corporate processes and usability engineering methodologies. Nonetheless, there is great potential for large corporations to contribute to open source projects. Providing usability know-how that leads to usable and useful products is a win-win situation for developers, the corporations, and – most importantly – the users.

David M Nichols, Michael B Twidale. "Usability Processes in Open Source Projects." Graduate School of Library and Information Science, University of Illinois at Urbana-Champaign, IL, USA.
In this paper we explore how open source projects address issues of usability. We describe the mechanisms, techniques and technology used by open source communities to design and refine the interfaces to their programs. In particular we consider how these developers cope with their distributed community, lack of domain expertise, limited resources and separation from their users. We also discuss how bug reporting and discussion systems can be improved to better support bug reporters and open source developers.

The "Professional Usability in Open Source Projects" article is quite short, and does not seem to contribute new material to usability in open source software. While an interesting find, I doubt this will be a useful reference to me.

However, the "Usability Processes in Open Source Projects" article has more useful material. In the next few days, I'll review this article further!

One possible candidate

At the risk of getting ahead of myself, one possible candidate for a usability case study is Calibre, an ebook management program. I use Calibre to add free ebooks to my Amazon Kindle ebook reader. Calibre supports all major platforms: Windows, Mac, and Linux.

Calibre uses icons, but it is not entirely icon-driven. Common actions are available on a toolbar with drop-down menus. The menus don't seem very deep, yet have moderate complexity. Overall, Calibre is geared for general users: anyone who wants to manage ebooks on their reader. The program is not too complex, yet not trivial. Calibre is Free / open source software, distributed under the GNU General Public License (version 3).

I'm not saying Calibre is the program I'll go with for a case study, but I'll certainly put it on my list of possibilities.

GNOME usability

When I proposed this study of usability in open source software, I was somewhat aware of usability studies of GNOME. Specifically, I knew that Sun Microsystems had once commissioned a usability study of an early version of GNOME (I think in the version 1.x series) as Sun incorporated the free GNOME desktop into Sun's Solaris operating system.

However, I didn't realize that the GNOME project continued this usability effort, and set up a GNOME Usability Project. From the website:
The Usability Project strives to make the GNOME experience as pleasant and efficient as possible. The project aims both to aid developers in their efforts to create intuitive applications, and to lead by creating designs and detailed mockups toward a cohesive and beautiful new generation of the GNOME desktop.

The Usability Project achieves these goals through the creation of an interface guide defining and evolving the GNOME user interface, working with maintainers to find existing interaction problems through user testing, and the visual design and interaction engineering of new desktop components.
In addition, the GNOME Usability Project sponsors occasional "usability hackfests" like the 2010 hackfest in London. Looking at the list of attendees, the GNOME usability hackfests include a wide array of contributors, both corporate and general users. A few corporate attendees you may recognize:

Over the next few days, I'll dive into the findings from the GNOME Usability Project and share some thoughts here. I tried to find the original GNOME usability study (from 2001) but that link no longer works.
It's important to keep in mind, however, that while the results of GNOME's usability study will be generally interesting as background material, a desktop environment such as GNOME would be too complex to use as a case study for my project. But the GNOME Usability Project can provide some feedback on my independent study. Certainly it is a rare case of an open source software project that considers usability in its program design.
Update (9/13/12):

Thanks to a link from Colin Walters's blog, I found the GNOME usability study:

Which program?

I have a workplan for each week in my independent study. I've reserved the three weeks from September 24 until October 14 to define which program I'd like to use as a case study.

While I might defer thinking about this problem, as it happens I've started to consider what factors would go into a good program for a case study. A few thoughts:

I want to avoid open source programs that serve a very limited audience. Typical examples in this space are programs written by developers for other developers. GNU Emacs is one such program: an extensible, customizable text editor that is mostly used by developers. Similarly, "integrated development environments" (IDEs), debuggers, and other programmers tools serve a limited, highly technical audience. Special tools such as FreeDOS XFDISK, while geared for all FreeDOS users, are limited in scope and would not provide an interesting usability case study.

Similarly, simple programs would yield very limited results, and likely would not apply to even moderately-sized programs. Therefore, I exclude general text editors such as FreeDOS EDIT and GNOME's gedit. Also: games and desk tools such as calculators.

On the other extreme, I would avoid programs that are overly complex or have very deep menus. I need to keep a manageable scope of this independent study, and programs like OpenOffice, LibreOffice, or even Firefox that have rich menus would expand the usability tests to a point where the results may not be applicable to other open source programs.

Icons are an important part of usability, and today's graphical programs often leverage some icon set to represent program actions. Typical examples are a "toolbar" of commonly-used activities, such as "copy" and "paste", or "bold" and "italics". I prefer to leave icons for possible inclusion in this study. That means I will avoid programs that exist solely in text mode, or other programs that do not make use of icons: FreeDOS XFDISK and EDIT (also mentioned above) and many file managers.

The right open source program for this usability case study would need to strike a careful balance: generally useful to a wide audience, not overly technical without becoming trivial, not too many menus, and some icons without being purely icon-driven.


I mentioned in my first post that my "output will be a publishable research paper that discusses the usability of open source software and the results of a case study." I've been thinking about the audience that I'd like to reach in my publication.

Who would be most interested in a usability study of open source software? Certainly, a journal focused on usability (such as JUS or UX) would be an excellent opportunity. However, while publishing an article in such a journal would be a great idea, I think the community or audience that would benefit from a usability study on open source software is the open source community itself. So as I consider options, I find myself leaning towards journals that serve the open source community.

There are several possible venues for a "deep dive" on the usability of open source software user interfaces. Some possible candidates, in approximate order of visibility and applicability:

Linux Journal
LJ is a front-runner among Linux and open source magazines. First published in 1994, LJ used to be a traditional subscription print journal, but recently (2011) switched to electronic only. LJ focuses mostly on Linux, but discusses open source projects in general. Articles published in LJ will see the widest distribution in the open source software community. Senior Editor: Doc Searls. Associate Editor: Shawn Powers.

Journal of Usability Studies
From its website: "The Journal of Usability Studies (JUS) is a peer-reviewed, international, online publication dedicated to promoting and enhancing the practice, research, and education of user experience design and evaluation." Ginny Redish and Jakob Nielsen are among several on the Advisory board. Co-Editors in Chief: Joe Dumas, Marilyn Tremaine.

UX Magazine
UX is an online magazine focused on topics about user experience. Managing Editor: Jonathan Anderson. Deputy Managing Editor: Josh Tyson.
An edited online magazine sponsored by Red Hat Software, this site is geared for all topics related to the open source community. While the site tends to avoid very technical details, an article about the usability of open source software would appear to be a welcome topic. community moderator: Jason Hibbets.

Red Hat Magazine
Despite its name, RHM is not specific to Red Hat Software or to Red Hat Linux. RHM addresses topics surrounding open source, software freedom, intellectual property, copyright, and education. An article about the usability of open source software would likely be a good topic, but may not see a wide distribution. Editorial team: Josh Gajownik, Bascha Harris, Jonathan Opp, Ruth Suehle, Adrienne Yancey.

Friday, September 7, 2012

Open Source Software & Usability

How does usability impact open source software development? How might usability be applied to open source software?

Welcome to my blog! I will do a "deep dive" on the usability of open source software user interfaces, a usability study of menus and icons in open source software, and examine how usability might be applied to open source software programs. This will involve a case study of a specific open source software program. The output will be a publishable research paper that discusses the usability of open source software and the results of a case study, including a proposed reorganization of menus.

I'll use this blog as an open forum to discuss topics related to open source software and usability. Feel free to leave your thoughts in the comments.