Wednesday, July 27, 2016

On discoverability

Daniel G. Siegel posted an item on his blog recently that is very closely tied to usability testing: On discoverability.

I've discussed elsewhere that usability is about real people doing real tasks in a reasonable amount of time. Some researchers also refer to "learnability" and "memorability" to define usability—this is very similar to discoverability. Can you discover the features of the system just by poking at it? Is the user interface obvious enough that you can figure it out on your own?

Daniel's post includes a video of a usability test that explores the Google search box on a smartphone. It's a short but very interesting video to watch, as the tester clearly becomes more and more frustrated with how to initiate a search.

I want to close by highlighting some of Daniel's comments:
if there is no way to discover what operations are possible just by looking at the screen and the interaction is numbed with no feedback by the devices, what's left? the interaction gets reduced to experience and familiarity where we only rely on readily transferred, existing skills.

And that's why usability is so important, especially for open source software where users often must learn the software with no tutorials or other instructions to guide them.

Tuesday, July 26, 2016

Testing for Usability

I recently came across a copy of Web Redesign 2.0: Workflow That Works (book, 2005) by Goto and Cotler. The book includes a chapter on "Testing for Usability" which is brief but informative. The authors comment that many websites are redesigned because customers want to add new feature or want to drive more traffic to the website. But they rarely ask the important questions: "How easy is it to use our website?" "How easily can visitors get to the information they want and need?" and "How easily does the website 'lead' visitors to do what you want them to do?" (That last question is interesting for certain markets, for example.)

The authors highlight this important attribute of usability: (p. 212)
Ease of use continues to be a top reason why customers repeatedly return to a site. It usually only takes one bad experience for a site to lose a customer. Guesswork has no place here; besides, you are probably too close to your site to be an impartial judge.

This is highly insightful, and underscores why I continue to claim that open source developers need to engage with usability. As an open source developer, you are very close to the project you are working on. You know where to find all the menu items, you know what actions each menu item represents, and you know how to get the program to do what you need it to do. This is obvious to you because you wrote the program. It probably made a lot of sense to you at the time to label a button or menu item the way you did. Will it make the same sense to someone who hasn't used the program before?

Goto and Cotler advise to test your current, soon-to-be-redesigned product. Testing the current system for usability will help you understand how users actually use it, and will indicate rough spots to focus on first.

The authors also provide this useful advice, which I quite like: (p. 215)
Testing new looks and navigation on your current site's regular users will almost always yield skewed results. As a rule, people dislike change. If a site is changed, many regular users will have something negative to say, even if the redesign is easier to use. Don't test solely on your existing audience.

Emphasis is mine. People dislike change. They will respond negatively to any change. So be cautious and include new users in your testing.

But how do you test usability? I've discussed several methods here before, including Heuristic Review, Prototype Test, Formal Usability Test, and Questionnaires. Similarly, Goto and Cotler recommend traditional usability testing, and suggest three general categories of testing: (p. 218)

TypeSettingTestersStyle
Informal testingMay take place in the tester's work environment or other setting.Co-workers or friends.Simple task list, observed and noted by a moderator.
Semiformal testingMay or may not take place in a formal test environment.Testers are pre-screened and selected.Moderator is usually a member of the team.
Formal testingTakes place in a formal facility.Testers are pre-screened and selected.Scenario tasks, moderated by a human factors specialist. May also include one-way mirror and video monitoring.


The authors also recommend building a task list that represents what real people actually would do, and writing a usability test plan (essentially, a brief document that describes your overall goals, typical users, and methodology). Goto and Cotler follow this with a discussion about screening test candidates, then conducting the session. I didn't see a discussion about how to write scenario tasks.

The role of the moderator is to remain neutral. When you introduce yourself as moderator, remind the tester that you will be a silent observer, and that you are testing the system, not them. Encourage the testers to "think aloud"—if they are looking for a "Print" button, they should say "I am looking for a 'Print' button." Don't describe the tasks in advance, and don't set expectations (such as "This is an easy task").

It can be hard to moderate a usability test, especially if you haven't done it before. You need to remain an observer; you cannot "rescue" a tester who seems stuck. Let them work it out for themselves. At the same time, if the tester has given up, you should move on to the next task.

Goto and Cotler recommend you compile and summarize data as you go; don't leave it all for the end. Think about your results while it is still fresh in your mind. The authors prefer a written report to summarize the usability test, showing the results of each test, problem areas, comments and feedback. As a general outline, they suggest: (p. 231)
  1. Executive summary
  2. Methodology
  3. Results
  4. Findings and recommendations
  5. Appendices

This is fine, but I prefer to summarize usability test results via a heat map. This is a simple visual device that concisely displays test results in a colored grid. Scenario tasks are on rows, and testers are on columns. For each cell, use green to represent a task that was easy for the tester to accomplish, yellow to represent a more difficult task, orange for somewhat hard, red for very difficult, and black for tasks the tester could not figure out.

Whatever your method, at the end of your test, you should identify those items that need immediate attention. Work on those, make improvements as suggested by the usability test, then test it again. The value in usability testing is to make it an iterative part of the design process: create a design, test it, update the design, test it again, repeat.
image: web-redesign.com

Thursday, July 21, 2016

Writing an ebook about usability?

I write more about this on my Coaching Buttons blog, that I'm thinking about writing an ebook. Actually, it's several ebooks. But the one that applies here is Open Source Usability.

Open Source Usability
In short, this would be an ebook about how to examine and improve the usability in free software and open source software. While many books exist about usability, none specifically focus on free and open source software. Intended for developers, this ebook will discuss what usability is, different ways to examine usability, how to measure usability, and ways to integrate usability test results back into the development stream to improve the next versions of the software.

Most of the material would likely be based on what I write here in my blog, but condensed and easier to learn from.

I might release this for free. If I publish through Amazon or Apple iTunes Bookstore or Google's Play Bookstore, I think I'd have to set the price to the minimum level, probably $1.

But writing an ebook is a lot of work. I'm not sure if people would really be that interested in an ebook like this. Would you read an ebook about Open Source Usability? Please let me know in the comments.

Wednesday, July 20, 2016

Responding with emoji

In Diana's usability test, she will moderate a "first experience" of GNOME. Testers will login to GNOME using a fresh "test" login, and go through the first-time experience. The testers will use a few scenario tasks to guide them through tasks that most users would usually do on a new computer (check email, copy files from a USB stick). Afterwards, Diana will interview each tester to see what they thought.

One question we are asking testers is to summarize different parts of the experience using an emoji. It's an interesting method I've seen applied to other user experience tests. The emoji is an abstract concept which lets the tester express their own feelings, but because we provide a set of emojis (testers don't make their own) it is easier for us to collate results.

A recent article in the Washington Post uses this method in their coverage of the political convention happening this week in the US. The Post presents a moment, and invites readers to click the emoji that represents their reaction:

click to view full size

If you click the "Show More" link, you get the full list of emoji (although the first few are repeated):

click to view full size

(The different numbers for the emoji is because I took the screenshots a few minutes apart, and the article updated the counts.)

This is the kind of result I hope to see in our "first experience" test. How do testers respond to their first experience with GNOME?
image: Smiley (Wikimedia, public domain)

Tuesday, July 19, 2016

How FreeDOS keeps DOS alive

I was interviewed recently by ComputerWorld Australia about several topics in free software / open source software. They recently ran an article about my work in FreeDOS: How (and why) FreeDOS keeps DOS alive. The article has also been picked up by Slashdot, LWN, and other places.

Maybe you already know about FreeDOS. If you don't, the brief history is this:
In 1994, I was an undergraduate physics student. Although I'd already discovered Linux, and even ran Linux dual-boot with DOS on my personal computer, I mostly used DOS for my day-to-day work. I really didn't like Windows—but if you remember Windows 3.x, this shouldn't be a surprise. So when Microsoft announced that "DOS is dead" and the next version of Windows would do away with DOS, I felt someone had to do something. And I decided to be that someone.

In June 1994, I announced a DOS project called "PD-DOS" (for "public domain DOS") that would replace MS-DOS, but we would make it available to everyone, including the source code! We soon renamed PD-DOS to FreeDOS, because so much of our code was under the GNU GPL, so was "Free software" and not "public domain."

We've continued to develop FreeDOS ever since. We are planning to release the FreeDOS 1.2 distribution "real soon now" for various values of "soon."
In related news, I did some digging into our early days of FreeDOS, and updated the Alpha 1-6 dates on our FreeDOS History page at the FreeDOS website, and the FreeDOS Road Map article at the FreeDOS wiki.
image: Official FreeDOS fish (cc-by)
by Bas Snabilie, adapted by Mateusz Viste
from FreeDOS Images

Friday, July 15, 2016

Reading RSS with Bash

Over at Linux Journal, I recently wrote an article about Parsing an RSS News Feed with a Bash Script.

My background with Linux started with Unix systems. I was a Unix systems administrator for several years before I introduced Linux at work. Over my career, I've learned to do a lot of cool things in shell scripts.

My LJ article idea originated with an update to the FreeDOS website. Like many other project websites, we fetch our news from a separate news system (in our case, from SourceForge) then parse it to display on the front page. Today, we do that every time someone loads the page. Effective, but inefficient. As I update the FreeDOS website, I wanted to automate the news feed to generate a static news "include" file, so I decided to do it in Bash.

Go read the LJ article for details, but the secret is in this Bash function:

xmlgetnext () {
   local IFS='>'
   read -d '<' TAG VALUE
}

That function reads a line of input until the next less-than (<) symbol, then breaks it into TAG and VALUE at the first greater-than (>) symbol. That makes it really simple to read well defined XML data like RSS, where the interesting data is always enclosed by opening and closing tags, like this:

<link>http://www.example.com/link/to/news/item/</link>

The LJ article describes step-by-step how to build a Bash script to parse an RSS feed. I'm using the script on our test site, and it's working well!
image: Linux Journal

Final preparations for usability tests

I wanted to take a moment to reflect on the schedule. A few weeks ago, I commented on moving to the project phase of usability testing.​ I included a calendar of our remaining work. If you refer back to the calendar, you will note we are nearing the end of week 8. Next week is week 9 (starting 18 July) so Diana, Renata and Ciarrai have another week to refine their usability tests.

Allan and I have shared some suggestions and edits for Diana, Renata and Ciarrai; they will take the next week to apply those edits and prepare the final version of their usability tests: opening script, scenario tasks, and any wrap-up questions.

Diana, Renata and Ciarrai are also starting to line up testers for each usability test. We know from research that you need at least five testers to get useful results as part of a tightly iterative process (create design, test it, update the design, test it again, etc) but since this test is a point in time, we are looking for ten test volunteers.

While the schedule says we'll start usability tests in week 10 (starting 25 July) there's no need to wait to start testing once everything is in order. Once Diana, Renata and Ciarrai have done their dry runs and are confident everything is ready, they may start their usability test at any time. So watch their blogs closely in the coming weeks for updates.

This is it!
image: Outreachy

Monday, July 11, 2016

Getting ready for usability tests

Since we have moved to the project phase in Outreachy usability testing, Diana, Ciarrai and Renata have been working on their usability tests. I wanted to share a reminder of the work that is going on:

Diana will examine a "user experience" of a user's first exposure to GNOME
In this test, Diana will ask testers to simulate an "unboxing" of a new system. The tester will turn on the laptop or computer, watch the computer start up, and login to a fresh "test" account so they get first-user experience.

The tester will experiment with the system, using a few scenarios (not scenario tasks) to suggest real tasks that real people might do, so the tester gets an opportunity to poke around and launch a few applications (such as web, music, files, etc). Since the tester will have a limited time to experiment with GNOME, we are suggesting three scenarios:

1. Documents, photos and music management

This makes a lot of sense for a first experience because new users will need to copy over their old data before they can use the computer. This is also a great opportunity to see how new users approach the folder structure: Documents, Downloads, Music, etc. Diana will also create a "forest" of sample files for the tester to work with: a bunch of music files, a bunch of photos, a bunch of documents in different formats (MS Word .doc or .docx, MS Excel .xls or .xlsx, LibreOffice/OpenOffice .odt and .ods, and plain text). We won't put them in the same folder names that GNOME uses, but will organize them both randomly (files of different types mixed in a folder like "My backup" or "Copies of old files") and structurally with different folder names (like "Songs" and "My files" and "Class papers").

2. Sending an email

This is also a good scenario because almost every person uses a computer to send email at some point. We will leave this scenario open-ended. For example, rather than specify the Evolution mail program, the scenario will provide a general direction like "After you start up your new computer, you want to check your email." I'm curious how people will interact with this. Gmail users and other webmail users may use the browser to access their email. Others may explore Evolution.

3. Setting up a browser

​This is another great first experience thing to do; almost everyone would likely use a web browser in the first few minutes of using a new computer. This scenario will encourage general use of the browser, such as asking the tester to visit a website like Google or their favorite news site.

After this brief experience, Diana will do a brief interview with each tester, asking questions like "Are you excited about it, or just interested, sort of neutral, or not that interested, or did you not like it?" Ask questions that uncovered how they approached the system. "Did it seem friendly in the brief time you got to use the desktop?" And the tester will sum up their first impression with an emoji to suggest their emotional response, and use a word association ("Describe this first experience in one word.")
Ciarrai will perform a paper prototype test of the new Settings app
The usability test with a paper prototype can be very similar to a traditional usability test with a final product. With the paper prototype test, we'll need to be careful about the test design.

The major change in the new GNOME Settings is this: "A major feature of the latest settings designs is a rethink of the GNOME Settings “shell” (that is, the overall framework of the settings application). We want to move from the current model, that uses an icon grid and fixed window size, to one that uses a list sidebar for navigation, and has a resizeable window."

Ciarrai has already written several draft scenarios for the prototype test, and now she is refining them into scenario tasks.

In the paper prototype test, Ciarrai will present these scenario tasks to the testers and ask them to indicate what list item they would pick. Remember that a scenario task sets up a brief context, then asks the tester to do something specific. For example: "You brought your laptop to a local cafe and you've already connected to the wireless network using the shop's shared wireless password, which was printed on your receipt but you've since thrown it in the trash. (Connecting to wireless networks happens outside Settings.) The person next to you asks how to get on the wireless network. Which Setting would you try first to find the wireless password so you can share it with the person sitting next to you?" (That one is kind of long.) Or "You're at the office and you want to show a work presentation with a projector. Which Setting would you try first to connect your laptop to the projector?"
Renata will perform a traditional usability test of other ongoing work in GNOME
This is basically the same usability test that applicants performed during their initial contribution to Outreachy.

In other cycles of Outreachy, we've started the traditional usability test analysis by looking at the design patterns that we'd like to evaluate, and building up the test based on patterns and features, matching the patterns/features of interest to the scenario tasks. But we've probably carried that as far as we can in previous usability tests. Sanskriti and Gina (and my own work) have uncovered design patterns that work well and others that need improvements. There's not much to be learned by repeating tests on design patterns that work well, so we will shift this test to look at those design patterns and design pattern implementations that have been improved in recent GNOME design iterations.

Renata has also performed her own analysis of GNOME design patterns.

Looking at Gina's usability test results, testers struggled to find the zoom button in Photos (header bar button), changing the month/year in Calendar (header bar buttons), searching (header bar button) and copying in Characters (primary window button), annotating and bookmarking in Evince (header bar menus), and other tasks in Nautilus (several were header bar menus). So the scenario tasks for this usability test will focus on Photos, Calendar, and Evince—with attention to header bar buttons and header bar menus (other design patterns too, but those two seem important).

This week, Renata is writing scenario tasks that exercise header bar buttons, primary window buttons, header bar menus and application menus. She will reference Gina's test and re-use any scenario tasks from her usability test that uncovered problems for testers. Re-using scenario tasks will allow us to better compare how design patterns have improved.

In each case, Diana, Ciarrai and Renata will also write a pre-test "script" to "set up" the test. You need to have something written down so you say the same thing to every tester. This script will inform the tester that our usability test will examine a free software desktop system. Something like: Just like Windows is a desktop system and MacOS is a desktop system, GNOME is a free software desktop system. And you'll be doing a few tasks using GNOME and a few GNOME applications. I'll give you the tasks one at a time. We aren't "testing" you. The test is all about the software. If you have problems with part of the test, that's okay; that's exactly what we are trying to find. So don't feel embarrassed if you have problems doing something. Also, I'm going to take notes while you're doing these tasks. It will help me if you talk out loud when you are doing something, so I can take notes. So if you're looking for a Print button, just say "I'm looking for a Print button." And move the mouse to wherever you are looking on the screen, so I can see where you're looking.

Diana, Ciarrai and Renata will also write their own wrap-up interview questions. These will likely be specific to each test, and they may ask some questions based on how each tester performed in the test (for example, "It looked like you had problems doing __ in the test; what were you looking for, or what would have made that more obvious.") But they will also need to have some questions about the test itself: What seemed hardest to do? What seemed easiest to do?

Watch in the next few days for updates!
image: Outreachy

Friday, July 1, 2016

Gmail vs Outlook

I go back quite a ways with email. I first started using email when I was an undergraduate, studying physics. Not a lot of people did email back then (early 1990s) but it was handy to stay connected with my brother who was studying computer science some 1,300 miles away from me.

Back then, I used whatever email client we had installed on the Sun Unix workstations in our computer lab. I think I started with standard Unix mail, then upgraded to pine.

After I graduated, I used email through various local service providers, or through work. I ran Linux at home by then, and I recall using a variety of local email clients, upgrading to whichever had more attractive features. Eventually, I settled on Netscape's Email program, which later became Mozilla and Firefox, then branched off to the standalone email client Thunderbird.

When I adopted GNOME as my default Linux desktop at home, I continued with Thunderbird for a while, then tried a few other GTK-enabled email programs before committing to GNOME's Evolution program. I even used Evolution at work, since I was fortunate to run Linux on my desktop at work.

Fast-forward to 2004, and Google released Gmail. While there had been a few "webmail" systems before that, Gmail was the first that really made "webmail" seem like you were using a desktop email client. Gmail was fast, easy, and great-looking. I switched my at-home email to Gmail around 2005 or 2006.

I worked for a university at the time, and we quickly realized our in-house email system couldn't compete with Gmail. We could only offer 25MB for each user's email storage, but Gmail offered 1GB. In 2005, Gmail expanded email storage to 2GB. That made us sit up and take notice of Gmail. As did others in higher ed. Email had truly become a commodity; we could no longer offer added value to our students, faculty, and staff through email systems we ran ourselves.

So it was in 2007 (I think ) that we launched a pilot initiative to migrate university email to Gmail. I wasn't in the pilot group, but I was first in line when we started moving general university accounts to Gmail. If I'm remembering my timeline correctly, I think I've been using Gmail at work since 2008.

And a few years after that, I seriously adopted my mobile phone (Android) and tablet (iPhone) as platforms for my email. Like probably many of you, I am equally likely to reply to email via my phone or laptop.

I've changed jobs, and we use Outlook at work. I use both the "OWA" web client and the desktop Outlook client. So there's another email client to add to my list.

Doing the math, I've been using email for over twenty-five years. Over that time, I've switched from one email system to another, using whatever had the best features, whatever was most interesting. I mention this by way of highlighting that I'm not a "who moved my cheese"-type of person. I am pretty fluid with technology. I use the technology or application that best suits my needs.

So it is with this perspective that I say this: I don't like Outlook for email. It just feels like a step backwards in how email clients should work in 2016.

Here's one example: tags.

In Gmail, you can tag email conversations. You're reading an email conversation, and you'd like to quickly refer back to it again later. You can tag the conversation with a colored label, also called a "tag," like audit or governance or security, whatever label and color appropriately represents the conversation.

In a way, you can think of tags as a kind of "folder" assignment. And the neat thing about these tags is they stick to the conversation whether or not the email conversation is in the inbox. This presents a very flexible workflow. You can move an email conversation to a folder, which assigns it a tag. If someone replies to the conversation, the whole conversation is resurrected to the inbox with the tag attached. When you're done reading the response, you click the Archive button and the conversation goes back into the folder.

You can assign multiple tags to a single conversation. A discussion about a security item in an audit might have both auditsecurity tags assigned, so you can later find the email conversation in both "audit" and "security" folders.

I really like that feature. I think Thunderbird added some kind of tags support, too, but this would have been after I stopped using Thunderbird.

But in Outlook? Tags don't exist. You can use "Categories" and assign a kind of label to something, but it's nothing like tags. So now it feels like with Outlook, I'm back in the Dark Ages of email. I can move a conversation to a folder—only one folder—but if someone replies to the conversation, the whole conversation reappears back in my inbox. And there's nothing to tell me where I'd put it previously.

At work I have two dozen email folders in Outlook. Most are project folders. And right now, I'm directing a work effort that spans two different projects. We have an active email thread going about the project work, and occasionally someone starts up a new discussion to talk about a different aspect of the work effort.

Even though the conversation relates to two different projects, I have to pick one folder when I archive the conversation. That's weird to me now. That would have been normal "back in the day," but in 2016 that seems primitive. When someone replies to the email discussion, the entire conversation appears back in my inbox, and I have to figure out which of the two projects to save it to. I can't store one email thread in two locations. Where did I store the last conversation? Did I put it in "ProjectA" or "ProjectB"?

This makes searches more difficult. If I need to uncover an earlier conversation about the project (and I do) I have to search across several folders. I can't just go to either of the tags and look there.

There are a few other examples, but if I'm honest with myself, tags are really the thing I miss. Yes, a few other things bug me because they are different from other email systems, but that's the nature of running a different email client.

Maybe tags exist in Outlook, and I don't know how to turn on that feature. Someone please tell me. Or maybe tags are a unique defining feature of Gmail and I shouldn't expect to find them elsewhere. I can just tell you that tags made my workflow so much easier, and I don't have them in Outlook.
images: Wikimedia (Outlook icon: public domain) (Gmail icon: public domain)