Sunday, February 2, 2014

About my research

The focus of my M.S. capstone project will be a usability test of GNOME. This will expand on my previous directed study in usability, when I conducted a usability test of a few programs (gedit, Nautilus, Firefox) under GNOME 3.4 in Fedora 17. The connections to academic study are strong here; there has been very little academic research about usability in open source software. Nichols and Twidale previously researched "Usability Processes in Open Source Projects." And Benson, Müller-Prove, & Mzourek published their results of "Professional Usability in Open Source Projects: GNOME,, NetBeans." GNOME has conducted a few usability tests. Otherwise, research and academia have not largely explored this area of practice.

And as suggested by Andersen (2013), academia and practice need to develop a reciprocal relationship. It's a cycle; academia needs to provide accessible, actionable research that is published in places visited by practice. That allows the practice to advance, which provides future research opportunity for academia. The cycle continues, and both academia and practice benefit.

The question of the research question, necessary for research, also plays a role in this cycle. Rude (2009) describes how a research question arises from a rhetorical exigence (an urgency) and the anticipation of possible solutions. So in conducting research on usability in open source software, we shouldn't simply evaluate existing systems, but also explore possible variations and improvements to those systems.

Therefore, as I plan my research, I want to ensure that my work will support and advance usability in open source software. That's the reciprocal relationship.

I want to expand my previous research on GNOME usability. In a video chat a few weeks ago, GNOME designer Allan Day described GNOME's new design patterns, which they are working to integrate in a future GNOME version. I might explore these new design patterns—so my research question is probably along the lines of "How well do users navigate/understand/… the new design patterns in GNOME?" A practical approach to this research question would be to measure the success/impact of these design patterns on typical behavior/activities. And usability can be measured; it is a quantitative practice. One analytical method is to generate a "heat map" that demonstrates the relative ease or difficulty in accomplishing certain scenarios or actions.

Day suggested a list of GNOME applications that use the new design patterns concept. I might use any of these in my usability test:

  • Clocks
  • Contacts
  • Documents
  • Gedit (development version only)
  • Maps
  • Music
  • Notes
  • Photos
  • Software
  • Web

One way to explore the research question would be to conduct a usability test against prototypes (functional or on-paper) of the updated applications, using the new design patterns. By testing several applications that use the same or similar design patterns, the results might identify areas where the design patterns work well vs where they do not work well. These results would inform future GNOME design and development (reciprocal relationship).


  1. Hi,

    Interesting research.

    I have a couple of questions:

    1. What is the title of the research work of Andersen (2013)?
    2. What is the title of the research work of Rude (2009)?

    I have a suggestion: At the end of post to include the reference of the research you mention.

  2. I haven't wanted to do that up 'til now because I was afraid it would make my blog too "academic-y" and I want this blog to be more accessible to everyone. But I'll try to include academic references where I can in future posts.

    Here are the references you asked about:

    - Rude, C.D. (2009). "Mapping the research questions in technical communication." JBTC, 23(2).

    - Andersen, R. (2013). "The value of a reciprocal relationship between research and practice." CIDM Information Management News.

  3. I'll add that usability in open source software isn't a topic that's well covered academically - there have been more articles & conference presentations in non-academic settings. Even so, usability IN GENERAL gets way more coverage than usability IN OPEN SOURCE SOFTWARE.

    A few references from academic spaces:

    • Andreasen, M. S.; Nielsen, H. V.; Schrøder, S. O.; Stage, J. (2006). “Usability in Open Source Software Development: Opinions and Practice.” ISSN 1392 – 124X Information Technology and Control, 2006, Vol.35, No.3A

    • Bach, Paula M. (2009). “Supporting the user experience in free/libre/open source software development.” PhD thesis, Pennsylvania State University. Retrieved from

    • Benson, Calum; Müller-Prove, Matthias; Mzourek, Jiri. (2004). “Professional Usability in Open Source Projects: GNOME,, NetBeans.” CHI EA '04 CHI '04 extended abstracts on Human factors in computing systems. pp 1083-1084. ACM New York.

    • Frishberg, N., Dirks, A. M., Benson, C., Nickell, S., Smith, S. (2002). “Getting to Know You: Open Source Development Meets Usability.” In: CHI 2002, Minneapolis, MI. ACM Press. Retrieved from and

    • Nichols, David M.; Twidale, Michael B. (2003). “The Usability of Open Source Software.” First Monday 8(1). Retrieved from

    • Nichols, David M.; Twidale, Michael B. “Usability Processes in Open Source Projects.” Graduate School of Library and Information Science, University of Illinois at Urbana-Champaign, IL, USA.

    • Nichols, David M.; Twidale, Michael B. (2005). “Exploring Usability Discussions in Open Source Development.” Proceedings, 38th Annual Hawaii International Conference on System Sciences, HICSS-38, Track 7: “Internet and the Digital Economy,” p.198c.

    • Trudelle, P. (2002). “Shall We Dance? Ten Lessons Learned from Netscape’s Flirtation with Open Source UI Development.” Web copy of CHI2002 article. Retrieved from

  4. Additionally, anyone interested in usability testing really should get this book:

    • Dumas, Joseph S. and Redish, Janice C. (1993. Reprinted 1994.) A Practical Guide to Usability Testing: Revised Edition. Portland, OR: Intellect.

  5. Hi,

    Thank you very much for sharing the literature.

    I would like to know where you get the information about: "there have been more articles & conference presentations in non-academic settings".

    It is a personal opinion?

  6. That comes from my lit review. I can't say I've found 100.00% of all papers on the subject, but I've got quite a lot of overlap so I think the list is pretty representative.

    1. Hi,

      Thanks for replying.

      What do you mean by overlap?