Monday, October 15, 2018

Making a first contribution in Outreachy usability testing

If you want to join us in GNOME usability testing as part of the upcoming cycle in Outreachy, you'll need to make a first contribution as part of your application process. Every project in Outreachy asks for a first contribution; this is a requirement in Outreachy.

Don't make too big of a deal about your first contribution in usability testing. We don't expect interns to know much about usability testing as they enter the internship. Throughout the internship, you'll learn about usability testing. So for this first contribution, we set a low bar.

GNOME is a desktop system, so the GNOME programs would be part of the desktop system. If you have installed a Linux distribution with the GNOME desktop, you can pick one or two GNOME programs that come installed as part of GNOME, and do a usability test on them. (If you haven't installed a Linux distribution, you can easily do so inside a PC emulator or virtual machine such as VirtualBox and install a desktop-focused Linux distribution like Fedora Workstation.)

Some GNOME programs you might pick from include:

  • gedit (GNOME text editor)
  • GNOME Web
  • GNOME Files
  • GNOME Software
  • GNOME Notes
  • Evince (GNOME PDF viewer)
  • GNOME Calendar
  • GNOME Photos

Or if you have a favorite GNOME program, you can do a usability test on that too.

A short usability test of 5 scenario tasks (I suggest 3 tasks for one program, and 2 tasks for another program) and 5 testers is a good first contribution. (Five testers may sound like a lot, but you can do your usability test with friends and family, which is completely valid.)

Here are some resources showing some usability test results with the above programs:


You can search for "scenario tasks" on the blog to help you write scenario tasks. Some of the blog articles there actually list some scenario tasks you can use, such as Ciarrai's first contribution from 2016 (skip ahead to Appendix/Scenarios for the scenario tasks Ciarrai used in their usability test contribution; you can just copy from there and that's fine). If you need help, let me know.

When you've done your usability test contribution, email me and we can arrange a conversation or discuss over email.

Sample Scenario Tasks

Here are six scenario tasks for gedit and four scenario tasks for GNOME Files. Feel free to pick and choose the tasks that you want to use in your usability test first contribution.

Note that the first scenario task for GNOME Files requires creating a file before the test. Don't forget to do that.

gedit (GNOME text editor)

1. You need to type up a quick note for yourself, briefly describing an event that you want to remember later. You start the Gedit text editor (this has been done for you).

Please type the following short paragraphs into the text editor:
Note for myself:

Jenny and I decided to throw a surprise party for Jeff, for Jeff's birthday. We'll get together at 5:30 on Friday after work, at Jeff's place. Jenny arranged for Jeff's roommate to keep him away until 7:00.

We need to get the decorations up and music ready to go by 6:30, when people will start to arrive. We asked everyone to be there no later than 6:45.
Save this note as party reminder.txt in the Documents folder.

2. After typing the note, you realize that you got a few details wrong. Please make these edits:

  • In the first paragraph, change Friday to Thursday.
  • Change 5:30 to 5:00.
  • Move the entire sentence Jenny arranged for Jeff's roommate to keep him away until 7:00. to the end of the second paragraph, after no later than 6:45.

When you are done, please save the file. You can use the same filename.

3. Actually, Jeff prefers to go by Geoff, and Jenny prefers Ginny. Please replace every occurrence of Jeff with Goeff, and all instances of Jenny with Ginny.

When you are done, please save the file. You can use the same filename.

4. You'd like to make a copy of the note, using a different name that you can find more easily later. Please save a copy of this note as Geoff surprise party.txt in the Documents folder.

For the purposes of this exercise, you do not need to delete the original file.

5. You decide the text in the editor is difficult to read, and you would prefer to use a different style of text. Please change the text to DejaVu Sans Mono, 12 point.

6. You decide the black-on-white text is too bright for your eyes, and you would prefer to use different colors. Please change the colors to the Oblivion color scheme.

GNOME Files

1. Yesterday, you re-organized your files and you don’t remember where you saved the copy of one of the articles you were working on. Please search for a file named The Hobbit.

2. Files and folders are usually displayed as icons, but you can display them in other ways too. Change how the file manager displays files and folders, to show them as a list.

3. You don’t have your glasses with you, so you aren’t able to read the names of the files and folders very well. Please make the text bigger, so it is easier to read.

4. Please search for a folder or a file that you recently worked on, maybe this will help you find the lost article.

Tuesday, October 9, 2018

Usability testing with Outreachy

I've volunteered with Allan and Jakub to mentor more GNOME usability testing in the next cycle of Outreachy, from December 4, 2018 to March 4, 2019. Outreachy expressly invites applicants from around the world who are women (both cis and trans), trans men, and genderqueer people.

Interns will work with the GNOME team and mentor(s) to do usability testing on GNOME. The goal is to perform several cycles of usability testing on prototypes of new designs, and provide usability testing results and feedback to the GNOME team so a new iterative design can be updated based on those results. We would like to use a "test what you've got" approach where we set up a testing schedule, and the intern tests whatever prototype or model is ready at that time. So if "test day" is Thursday, we could nail down what to test by Monday, and have the intern post results on Friday or the weekend.

What you need to know

You don't need to know usability testing to intern with us! We'll start the internship with a few weeks of "usability 101" with the mentor to learn about usability testing and usability processes, including how to lead a usability test. After the first few weeks, interns should expect to learn as they go, or "learn by doing."

This "learn by doing" approach will help you gain practical experience about usability testing. By the end of the internship, you will understand usability testing, and why it is important in open source software (this applies to any open source software project).

As an intern, you can have a direct impact on the GNOME desktop! Your results will help us improve the next versions of GNOME, and make GNOME easier for everyone to use.

I'll also note that in previous cycles of Outreachy, many of the interns have done such great work that I co-authored articles with them. If your tests are successful and generate useful results, we may co-author an article with you, and submit it to a technology/open source magazine or news website, or (non-academic) journal. That's a big bonus to your resume!

How to start

If you're interested in joining us for GNOME usability testing, you first need to apply for an internship at Outreachy. You'll also need to make a first contribution.

You can do your first contribution by performing a usability test of your own, using GNOME, since that's the topic of the usability testing internship. You can pick one or a few programs from GNOME, and do a total of ten scenario tasks.

When you've finished your usability test, generate a heat map of your results, then write a brief summary about the experience. This is not a formal paper, it's something you would write on a blog or send in an email. Please use an informal voice here; imagine you are explaining your test to a friend. Email your summary to me (jhall@gnome.org) in OTF or PDF format, and I'll review the results with you. Or if you prefer, you can publish your results on your blog and email the URL to me.