Saturday, December 8, 2018

Week 1 of GNOME usability testing

The Outreachy internship started this week! For this cycle, we are joined by Clarissa, who will help us with usability testing in GNOME.

I wanted to share our progress in the internship. I hope to provide regular status updates on our work.

For this week, I want to start with a grounding in Usability testing. You can do usability testing in different ways. Alice Preston described the types of usability methods for the STC newsletter (Vol 10, No. 3). I can't find a copy of that newsletter online, but in the list, she describes 11 ways to evaluate usability in a design. Preston's list is:

  1. Interviews/Observations: One-on-one sessions with users.
  2. Focus Groups: Often used in marketing well before there is any kind of prototype or product to test, a facilitated meeting with multiple attendees from the target user group.
  3. Group Review or Walk-Through: A facilitator presents planned workflow to multiple attendees, who present comments on it.
  4. Heuristic Review: Using a predefined set of standards, a professional usability expert reviews someone else's product or product design and presents a marked checklist back to the designer.
  5. Walk-Around Review: Copies of the design/prototype/wireframe are tacked to the walls, and colleagues are invited to comment.
  6. Do-it-Yourself Walk-Through: Make mock-ups of artifacts, but make the scenarios realistic. Walk through the work yourself.
  7. Paper Prototype Test: Use realistic scenarios but a fake product.
  8. Prototype Test: A step up from a paper prototype, using some type of animated prototype with realistic scenarios.
  9. Formal Usability Test: Using a stable product, an animated prototype, or even a paper prototype, test a reasonably large number of subjects against a controlled variety of scenarios.
  10. Controlled Experiment: A comparison of two products, with careful statistical balancing, etc
  11. Questionnaires: Ask testers to complete a formal questionnaire, or matching questionnaire.


Most of our work in the internship will be testing designs that haven't gone "live" yet (this is called "prototype usability testing"). Allan and Jakub will create mock-ups of new designs, and Clarissa will do usability testing on them. You can read a bit about this on Allan's blog post.

In that post, Allan writes:
Therefore, for this round of the Outreachy internships, we are only going to test UX changes that are actively being worked on. Instead of testing finished features, the tests will be on two things:
  1. Mockups or prototypes of changes that we hope to implement soon (this can include static mockups and paper or software prototypes)
  2. Features or UI changes that are being actively worked on, but haven’t been released to users yet

Ciarrai interned with us a few years ago, and worked on prototype usability testing for the then-upcoming GNOME Settings app redesign. We wrote an article together for OpenSource.com about paper-based usability testing that you might find interesting.

You may also be interested in an older article I wrote about usability testing with prototypes. I don't expect that we'll be testing with the animated prototypes that my article proposes, but it's good background material.

As you prepare for usability testing, you may ask "How many testers do I need?" The answer is "about five" if you are doing a traditional usability test. For a paper prototype test, this may be different. How many do you think we need?

For additional background, you might be interested to read this essay about how many testers you need, and why the answer is "about five." Note that "five" only applies to doing usability testing in a traditional or "formal" usability test, where you test real people with real scenario tasks against a real product. For paper prototype testing, you have a different value for L, so we'll need a different number of testers. But this is a good start to understanding the assumptions.

Usability expert Jakob Nielsen has a good video about "five testers" that I often recommend.

Looking for more background information? Here are a few additional articles related to "how many testers do I need?"



That last article makes an interesting point that I'll quote here: "Studies to evaluate a prototype of a novel user-interface design often concern the discovery of severe show-stopper problems. Testing usually reveals such severe errors relatively quickly, so these tests often require fewer participants."

But, how many testers do you think you'll need for each iteration of prototype testing?

Monday, November 12, 2018

Why the Linux console has sixteen colors (SeaGL)

At the 2018 Seattle GNU/Linux Conference after-party, I gave a lightning talk about why the Linux console has only sixteen colors. Lightning talks are short, fun topics. I enjoyed giving the lightning talk, and the audience seemed into it, too. So I thought I'd share my lightning talk here. These are my slides in PNG format, with notes added:
Also, my entire presentation is under the CC-BY:
When you bring up a terminal window, or boot Linux into plain text mode, maybe you've wondered why the Linux console only has sixteen colors. No matter how awesome your graphics card, you only get these sixteen colors for text:
You can have eight background colors, and sixteen foreground colors. But why is that?
Remember that Linux is a PC operating system, so you have to go back to the early days of the IBM PC. Although the rules are the same for any Unix terminal.

The origins go back to CGA, the Color/Graphics Adapter from the earlier PC-compatible computers. This was a step up from the plain monochrome displays; as the name implies, monochrome could only display black or white. CGA could display a limited range of colors.

CGA supported mixing red (R), green (G) and blue (B) colors. In its simplest form, RGB is either "on" or "off." In this case, you can mix the RGB colors in 2×2×2=8 ways. So RGB=100 is Red, and RGB=010 is Green, and RGB=001 is Blue. And you can mix colors, like RGB=011 is cyan. This simple table shows the binary and decimal representations of RGB:
To double the number of colors, CGA added an extra bit called the "intensifier" bit. With the intensifier bit set, the red, green and blue colors would be set to their maximum values. Without the intensifier bit, each RGB value would be set to a "midrange" intensity. Let's represent that intensifier bit as an extra 1 or 0 in the binary color representation, as iRGB:
That means 0100 gives "red" and 1100 (with intensifier bit set) results in "bright red." Also, 0010 is "green" and 1010 is "bright green." And 0000 is "black," but 1000 is "bright black."

Oh wait, there's a problem. "Black" and "bright black" are the same color, because there's no RGB value to intensify.

But we can solve that! CGA actually implemented a modified iRGB definition, using two intermediate values, at about one-third and two-thirds intensity. Most "normal" mode (0–7) colors used values at the two-thirds intensity. Translating from "normal" mode to "bright" mode, convert zero values to the one-third intensity, and two-thirds values to full intensity.

With that, you can represent all the colors in the rainbow: red, orange, yellow, blue, indigo, and violet. You can sort of fake the blue, indigo, and violet with the different "blue" shades.

Oops, we don't have orange! But we can fix that by assigning 0110 yellow a one-third green value that turned the color into orange, although most people saw it as brown.

Here's another iteration of the color table, using 0x0 to 0xF for the color range, with 0x5 and 0xA as the one-third and two-thirds intensities, respectively:
And that's how the Linux console got sixteen text colors! That's also why you'll often see "brown" labeled "yellow" in some references, because it started out as plain "yellow" before the intensifier bit. Similarly, you may also see "gray" represented as "bright black," because "gray" is really "black" with the intensifier bit set.

So let's look at the bit patterns. You have four bits for the foreground color, 0000 black to 1111 bright white:
And you have three bits for the background color, from 000 black to 111 white:
But why not four bits for the background color? That's because the final bit is reserved for a special attribute. With this attribute set, your text could blink on and off. The "Blink" bit was encoded at the end of the foreground and background bit-pattern:
That's a full byte! And that's why the Linux console has only sixteen colors; the Linux console inherits text mode colors from CGA, which encodes colors a full byte at a time.

It turns out the rules are the same for other Unix terminals, which also used eight bits to represent colors. But on other terminals, 0110 really was yellow, not orange or brown.

Usability Testing in Open Source Software (SeaGL)

I recently attended the 2018 Seattle GNU/Linux Conference, where I gave a presentation about usability testing in open source software. I promised to share my presentation deck. Here are my slides in PNG format, with notes added:
Also, my entire presentation is under the CC-BY:
I've been involved in Free/open source software since 1993, but recently I developed an interest in usability testing in open source software. During a usability testing class in my Master's program in Scientific and Technical Communication (MS) I studied the usability of GNOME and Firefox. Later, I did a deeper examination of the usability of open source software, focusing on GNOME, as part of my Master's capstone. (“Usability Themes in Open Source Software,” 2014.)

Since then, I've joined the GNOME Design Team where I help with usability testing.

I also (sometimes) teach usability at the University of Minnesota. (CSCI 4609 Processes, Programming, and Languages: Usability of Open Source Software.)
I’ve worked with others on usability testing since then. I have mentored in Outreachy, formerly the Outreach Program for Women. Sanskriti, Gina, Renata, Ciarrai, Diana were all interns in usability testing. Allan and Jakub from the GNOME Design Team co-mentored as advisers.
What do we mean when we talk about “usability”? You can find some formal definitions of usability that talk about the Learnability, Efficiency, Memorability, Errors, and Satisfaction. But I find it helps to have a “walking around” definition of usability.

A great way to summarize usability is to remember that real people are busy people, and they just need to get their stuff done. So a program will have good usability if real people can do real tasks in a realistic amount of time.

User eXperience (UX) is technically not the same as usability. Where usability is about real people doing real tasks in a reasonable amount of time, UX is more about the emotional connection or emotional response the user has when using the software.

You can test usability in different ways. I find the formal usability test and prototype test work well. You can also indirectly examine usability, such as using an expert to do a heuristic evaluation, or using questionnaires. But really, nothing can replace watching a real person trying to use your software; you will learn a lot just by observing others.
People think it's hard to do usability testing, but it's actually easy to do a usability test on your own. You don’t need a fancy usability lab or any professional experience. You just need to want to make your program easier for other people to use.

If you’re starting from scratch, you really have three steps to do a formal usability test:

1. Consider who are your users. Write this down as a short paragraph for each kind of user for your software. Make it a realistic fiction. These are your Personas. With personas, you can make design decisions that always benefit the user. “If we change __ then that will make it easier for users like Jane.” “If we add __ then that will help people like Steve.”

2. For each persona, write a brief statement about why that user might use the software to do their tasks. There are different ways that a user might use the software, but just jot down one way. This is a Use Scenario. With scenarios, you can better understand the circumstances when people use the software.

3. Now take a step back and think about the personas and scenarios. Write down some realistic tasks that real people would do with the software. Make each one stand on its own. These are scenario tasks, and they make up your actual usability test. Where you should write personas and scenarios in third-person (“__ does this..”) you should write scenario tasks in second-person (“you do this..”) Each scenario task should set up a brief context, then ask the tester to do something specific. For example:

You don’t have your glasses with you, so it’s hard to see the text on the screen. Make the text bigger so you can read it more easily.

The challenge in scenario tasks is not to accidentally give hints for what the tester should do. Avoid using the same words and phrases from menus. Don’t be too exact about what the tester should do - instead, describe the goal, and let the tester find their own path. Remember that there may be more than one way to do something.

The key in doing a usability test is to make it iterative. Do a usability test, analyze your results, then make changes to the design based on what you learned in the test. Then do another test. But how many testers do you need?
You don’t need many testers to do a usability test if you do it iteratively. Doing a usability test with five testers is enough to learn about the usability problems and make tweaks to the interface. At five testers, you’ve uncovered more than 80% of usability problems, assuming most testers can uncover about 31% of issues (typical).

But you may need more testers for other kinds of usability tests. “Only five” works well for traditional/formal usability tests. For a prototype test, you might need more testers.

But five is enough for most tests.
If every tester can uncover about 31% of usability problems, then note what happens when you have one, five, and ten testers in a usability test. You can cover 31% with one tester. With more testers, you have overlap in some areas, but you cover more ground with each tester. At five testers, that’s pretty good coverage. At ten testers, you don’t have considerably better coverage, just more overlap.

I made this sample graphic to demonstrate. The single red square covers 31% of the grey square's area (in the same way a tester can usually uncover about 31% of the usability problems, if you've designed your test well). Compare five and ten testers. You don't get significantly more coverage at ten testers than at five testers. You get some extra coverage, and more overlap, but that's a lot of extra effort for not a lot of extra value. Five is really all you need.
Let me show you a usability test that I did. Actually, I did two of them. This was part of my work on my Master’s degree. My capstone was Usability Themes in Open Source Software. Hall, James. (2014). Usability Themes in Open Source Software. University of Minnesota.

I wrote up the results for each test as separate articles for Linux Journal: “The Usability of GNOME” (December, 2014) and “It’s about the user: Usability in open source software” (December, 2013).
I like to show results in a “heat map.” A heat map is just a convenient way to show test results. Scenario tasks are in rows and each tester is a separate column.

For each cell (a tester doing a task) I use a color to show how easy or how difficult that task was for the tester. I use this scale:

Green if the tester easily completed the task. For example, if the tester seemed to know exactly what to do, what menu item to activate or which icon to click, you would code the task in green for that tester.

Yellow if the tester experienced some (but not too much) difficulty in the task.

Orange if the tester had some trouble in the task. For example, if the tester had to poke around the menus for a while to find the right option, or had to hunt through toolbars and selection lists to locate the appropriate icon, you would code the task in orange for that tester.

Red if the tester experienced severe difficulty in completing the task.

Black if the tester was unable to figure out how to complete the task, and gave up.

There are some “hot” rows here, which show tasks that were difficult for testers: setting the font and colors in gedit, and setting a bookmark in Nautilus. Also searching for a file in Nautilus was a bit challenging, too. So my test recommended that the GNOME Design Team focus on these four to make them easier to do.
This next one is the heat map from my capstone project.

Note that I tried to do a lot here. You need to be realistic in your time. Try for about an hour (that’s what I did) but make sure your testers have enough time. The gray “o” in each cell is where we didn’t have enough time do that task.

You can see some “hot rows” here too: setting the font in gedit, and renaming a folder in Nautilus. And changing all instances of some words in gedit, and installing a program in Software, and maybe creating two notes in Notes.
Most of the interns did a traditional usability test. So that’s what Sanskriti did here:
Sanskriti did a usability test that was similar to mine, so we could measure changes. She had a slightly different color map here, using two tones for green. But you can see a few hot rows: changing the default colors in gedit, adding photos to an album in Photos, and setting a photo as a desktop wallpaper from Photos. Also some warm rows in creating notes in Notes, and creating a new album in Photos.
Gina was from my second cycle in Outreachy, and she did another traditional usability test:
You can see some hot rows in Gina's test: bookmarking a location in Nautilus, adding a special character (checkmark) using Characters and Evince, and saving the location (bookmark) in Evince. Also some warm rows: changing years in Calendar, and saving changes in Evince. Maybe searching for a file in Nautilus.

Gina did such great work that we co-authored an article in FOSS Force: "A Usability Study of GNOME" (March, 2016).
In the next cycle of Outreachy, we had three interns: Gina, Ciarrai and Diana. Renata did a traditional usability test:
In Renata’s heat map, you can see some hot rows: creating an album in Photos, adding a new calendar in Calendar, and connecting to an online account in Calendar. And maybe deleting a photo in Photos and setting a photo as a wallpaper image in Photos. Some issues in searching for a date in Calendar, and creating an event in Calendar.

See also our article in Linux Voice Magazine: "GNOME Usability Testing" (November, 2016, Issue 32).
Ciarrai did a prototype test for a future design change to the GNOME Settings application:
In the future Settings, the Design Team thought they’d have a list of categories down the side. Clicking on a category shows you the settings for that category. Here’s a mock-up for Wi-Fi in the new Settings. You can see the list of other categories down the left side:
Remember the “only five” slide from a while back? That’s only for traditional/formal usability tests. For a prototype test, we didn’t think five was enough, so Ciarrai did ten testers.

For Ciarrai’s heat map, we used slightly different colors because the tester wasn’t actually using the software. They were pointing to a paper printout. Here, green indicates the tester knew exactly what to point to, and red indicates they pointed to the wrong one. Or for some tasks that had sub-panels, orange indicates they got to the first panel, and failed to get to the second setting.

You can see some hot rows, indicating where people didn’t know what category would have the Settings option they were looking for: Monitor colors, and Screen lock time. Also Time zone, Default email client, and maybe Bluetooth and Mute notifications.
Other open source projects have adopted the same usability test methods to examine usability. Debian did a usability test of GNOME. Here’s their test: (*original)
They had more general “goals” for testers, called “missions.” Similar to scenario tasks, the missions had a more broad goal that provided some flexibility for the tester. But not very different from scenario tasks.

You can see some hot rows here: temporary files and change default video program in Settings, and installing/removing packages in Package Management. Also some issues in creating a bookmark in Nautilus, and adding/removing other clocks in Settings.
If you want more information, please visit my blog or email me.
I hope this helps you to do usability testing on your own programs. Usability is not hard! Anyone can do it!

Monday, October 15, 2018

Making a first contribution in Outreachy usability testing

If you want to join us in GNOME usability testing as part of the upcoming cycle in Outreachy, you'll need to make a first contribution as part of your application process. Every project in Outreachy asks for a first contribution; this is a requirement in Outreachy.

Don't make too big of a deal about your first contribution in usability testing. We don't expect interns to know much about usability testing as they enter the internship. Throughout the internship, you'll learn about usability testing. So for this first contribution, we set a low bar.

GNOME is a desktop system, so the GNOME programs would be part of the desktop system. If you have installed a Linux distribution with the GNOME desktop, you can pick one or two GNOME programs that come installed as part of GNOME, and do a usability test on them. (If you haven't installed a Linux distribution, you can easily do so inside a PC emulator or virtual machine such as VirtualBox and install a desktop-focused Linux distribution like Fedora Workstation.)

Some GNOME programs you might pick from include:

  • gedit (GNOME text editor)
  • GNOME Web
  • GNOME Files
  • GNOME Software
  • GNOME Notes
  • Evince (GNOME PDF viewer)
  • GNOME Calendar
  • GNOME Photos

Or if you have a favorite GNOME program, you can do a usability test on that too.

A short usability test of 5 scenario tasks (I suggest 3 tasks for one program, and 2 tasks for another program) and 5 testers is a good first contribution. (Five testers may sound like a lot, but you can do your usability test with friends and family, which is completely valid.)

Here are some resources showing some usability test results with the above programs:


You can search for "scenario tasks" on the blog to help you write scenario tasks. Some of the blog articles there actually list some scenario tasks you can use, such as Ciarrai's first contribution from 2016 (skip ahead to Appendix/Scenarios for the scenario tasks Ciarrai used in their usability test contribution; you can just copy from there and that's fine). If you need help, let me know.

When you've done your usability test contribution, email me and we can arrange a conversation or discuss over email.

Sample Scenario Tasks

Here are six scenario tasks for gedit and four scenario tasks for GNOME Files. Feel free to pick and choose the tasks that you want to use in your usability test first contribution.

Note that the first scenario task for GNOME Files requires creating a file before the test. Don't forget to do that.

gedit (GNOME text editor)

1. You need to type up a quick note for yourself, briefly describing an event that you want to remember later. You start the Gedit text editor (this has been done for you).

Please type the following short paragraphs into the text editor:
Note for myself:

Jenny and I decided to throw a surprise party for Jeff, for Jeff's birthday. We'll get together at 5:30 on Friday after work, at Jeff's place. Jenny arranged for Jeff's roommate to keep him away until 7:00.

We need to get the decorations up and music ready to go by 6:30, when people will start to arrive. We asked everyone to be there no later than 6:45.
Save this note as party reminder.txt in the Documents folder.

2. After typing the note, you realize that you got a few details wrong. Please make these edits:

  • In the first paragraph, change Friday to Thursday.
  • Change 5:30 to 5:00.
  • Move the entire sentence Jenny arranged for Jeff's roommate to keep him away until 7:00. to the end of the second paragraph, after no later than 6:45.

When you are done, please save the file. You can use the same filename.

3. Actually, Jeff prefers to go by Geoff, and Jenny prefers Ginny. Please replace every occurrence of Jeff with Goeff, and all instances of Jenny with Ginny.

When you are done, please save the file. You can use the same filename.

4. You'd like to make a copy of the note, using a different name that you can find more easily later. Please save a copy of this note as Geoff surprise party.txt in the Documents folder.

For the purposes of this exercise, you do not need to delete the original file.

5. You decide the text in the editor is difficult to read, and you would prefer to use a different style of text. Please change the text to DejaVu Sans Mono, 12 point.

6. You decide the black-on-white text is too bright for your eyes, and you would prefer to use different colors. Please change the colors to the Oblivion color scheme.

GNOME Files

1. Yesterday, you re-organized your files and you don’t remember where you saved the copy of one of the articles you were working on. Please search for a file named The Hobbit.

2. Files and folders are usually displayed as icons, but you can display them in other ways too. Change how the file manager displays files and folders, to show them as a list.

3. You don’t have your glasses with you, so you aren’t able to read the names of the files and folders very well. Please make the text bigger, so it is easier to read.

4. Please search for a folder or a file that you recently worked on, maybe this will help you find the lost article.

Tuesday, October 9, 2018

Usability testing with Outreachy

I've volunteered with Allan and Jakub to mentor more GNOME usability testing in the next cycle of Outreachy, from December 4, 2018 to March 4, 2019. Outreachy expressly invites applicants from around the world who are women (both cis and trans), trans men, and genderqueer people.

Interns will work with the GNOME team and mentor(s) to do usability testing on GNOME. The goal is to perform several cycles of usability testing on prototypes of new designs, and provide usability testing results and feedback to the GNOME team so a new iterative design can be updated based on those results. We would like to use a "test what you've got" approach where we set up a testing schedule, and the intern tests whatever prototype or model is ready at that time. So if "test day" is Thursday, we could nail down what to test by Monday, and have the intern post results on Friday or the weekend.

What you need to know

You don't need to know usability testing to intern with us! We'll start the internship with a few weeks of "usability 101" with the mentor to learn about usability testing and usability processes, including how to lead a usability test. After the first few weeks, interns should expect to learn as they go, or "learn by doing."

This "learn by doing" approach will help you gain practical experience about usability testing. By the end of the internship, you will understand usability testing, and why it is important in open source software (this applies to any open source software project).

As an intern, you can have a direct impact on the GNOME desktop! Your results will help us improve the next versions of GNOME, and make GNOME easier for everyone to use.

I'll also note that in previous cycles of Outreachy, many of the interns have done such great work that I co-authored articles with them. If your tests are successful and generate useful results, we may co-author an article with you, and submit it to a technology/open source magazine or news website, or (non-academic) journal. That's a big bonus to your resume!

How to start

If you're interested in joining us for GNOME usability testing, you first need to apply for an internship at Outreachy. You'll also need to make a first contribution.

You can do your first contribution by performing a usability test of your own, using GNOME, since that's the topic of the usability testing internship. You can pick one or a few programs from GNOME, and do a total of ten scenario tasks.

When you've finished your usability test, generate a heat map of your results, then write a brief summary about the experience. This is not a formal paper, it's something you would write on a blog or send in an email. Please use an informal voice here; imagine you are explaining your test to a friend. Email your summary to me (jhall@gnome.org) in OTF or PDF format, and I'll review the results with you. Or if you prefer, you can publish your results on your blog and email the URL to me.

Friday, September 28, 2018

Open source tools I used to write my latest book

I first used and contributed to Free software and open source software in 1993, and since then I've been an open source software developer and evangelist. I've written or contributed to dozens of open source software projects, although the one that I'll be remembered for is the FreeDOS Project, an open source implementation of the DOS operating system.

I recently wrote a book about FreeDOS. Using FreeDOS is my celebration of the 24th anniversary of FreeDOS. This is a collection of how-tos about installing and using FreeDOS, essays about my favorite DOS applications, and quick-reference guides to the DOS command line and DOS batch programming. I've been working on this book for the last few months, with the help of a great professional editor.

Using FreeDOS is available under the Creative Commons Attribution (cc-by) International Public License. You can download the EPUB and PDF versions at no charge from the FreeDOS e-books website. (There's also a print version, for those who prefer a bound copy.)

The book was produced almost entirely with open source software. I'd like to share a brief insight into the tools I used to create, edit, and produce Using FreeDOS.

Google Docs
This was the only tool that wasn't open source software. I uploaded my first drafts to Google Docs so my editor and I could collaborate. I'm sure there are open source collaboration tools, but the ability for two people to edit the same document at the same time, comments, edit suggestions, change tracking—not to mention the use of paragraph styles and the ability to download the finished document—made Google Docs a valuable part of the editing process.
LibreOffice
I started on LibreOffice 6.0 but I finished the book using LibreOffice 6.1. I love LibreOffice's rich support of styles. Paragraph styles made it really easy to apply a style for titles, headers, body text, sample code, and other text. Character styles let me modify the appearance of text within a paragraph, such as inline sample code or a different style to indicate a filename. Graphics styles let me apply certain styling to screenshots and other images. And page styles allowed me to easily modify the layout and appearance of the page.
GIMP
My book includes a lot of DOS program screenshots, website screenshots, and FreeDOS logos. I used the GIMP to modify these images for the book. Usually this was simply cropping or resizing an image, but as I prepare the print edition of the book, I'm using the GIMP to create a few images that will be simpler for print layout.
Inkscape
Most of the FreeDOS logos and fish mascots are in SVG format, and I used Inkscape for any image tweaking here. And in preparing the PDF version of the ebook, I wanted a simple blue banner at top of the page, with the FreeDOS logo in the corner. After some experimenting, I found it easier to create an SVG image in Inkscape that looked like the banner I wanted, and pasted that into the header.
ImageMagick
While it's great to use GIMP to do the fine work, sometimes it's just faster to run an ImageMagick command over a set of images, such as to convert into PNG format or to resize images.
Sigil
LibreOffice can export directly to EPUB format, but it wasn't a great transfer. I haven't tried creating an EPUB with LibreOffice 6.1, but LibreOffice 6.0 didn't include my images. It also added styles in a weird way. I used Sigil to tweak the EPUB file and make everything look right. Sigil even has a preview function so you can see what the EPUB will look like.
QEMU
Because this book is about installing and running FreeDOS, I needed to actually run FreeDOS. You can boot FreeDOS inside any PC emulator, including VirtualBox, QEMU, GNOME Boxes, PCem, Bochs. But I like the simplicity of QEMU. And the QEMU console lets you issue a screendump in PPM format, which is ideal for grabbing screenshots to include in the book.

And of course, I have to mention running GNOME on Linux. I use the Fedora distribution of Linux.
This article originally appeared on Opensource.com as 6 open source tools for writing a book.

Wednesday, August 22, 2018

Getting back into Outreachy

Outreachy is a great organization that helps women and other minorities get involved in open source software. (Outreachy was formerly the GNOME Outreach Program for Women.) I've mentored several cycles in Outreachy, doing usability testing with GNOME. I had a wonderful time, and enjoyed working with all the talented individuals who did usability testing with us.

I haven't been part of Outreachy for a few years, since I changed jobs. I have a really hectic work schedule, and the timing hasn't really worked out for me. Outreachy recently posted their call for participation in the December-March cycle of Outreachy. December to March should be a relatively stable time on my calendar, so this is looking like a great time to get involved again.

I don't know if GNOME plans to hire interns for the upcoming cycle of Outreachy, at least for usability testing. But I am interested in mentoring if they do.

Following conversations with Allan Day and Jakub Steiner, from GNOME Design, I'm thinking about changing the schedule we would use in usability testing. In previous cycles, I set up the schedule like a course on usability. That was a great learning experience for the interns, as they had a ramp-up in learning about usability testing before we did a big usability project.

But Allan and I would like to get interns involved more quickly. Also, Allan would prefer to have testing be more integrated to a current design project. Allan has a great point in saying, "rather than releasing something and having it tested weeks or months later, we need to be testing what we are working on right now."

My idea for the next cycle is to accelerate usability testing! I would imagine restructuring the internship so applicants would do a "crash course" in usability concepts the first week, then start doing usability tests immediately after that. Learn by doing!

For a 13 week internship, I imagine an intern or interns could spend almost the whole time leading usability tests, especially focusing on "you need five testers for an iterative process" so they are doing basically the same test throughout the internship, against different iterations of different designs. That would provide more immediate feedback to designs you are working on at the time.

A possible schedule could be:
  1. Quick study of usability testing background and methods
  2. Review test needs with Design Team; plan test #1
  3. Execute test #1
  4. Discuss results with Design Team
  5. Plan test #2
  6. Execute test #2
  7. Discuss results with Design Team
  8. Plan test #3
  9. Execute test #3
  10. Discuss results with Design Team
  11. Plan test #4
  12. Execute test #4
  13. Discuss results with Design Team; wrap-up

Based on that schedule, we could fit 4 usability test iterations in 13 weeks. We might do as many as 5 if things accelerate toward the end, such as shorter turnaround in refining the design. This depends on the time required to generate a new design based on the input of the usability tests.

The key would be to connect the design process and timeline to the internship timeline. And then to be realistic in testing. For example, we might have the intern do a paper prototype test one week, then spend the next week discussing results and figuring out the next iteration, and the intern does another paper prototype test, etc.

Paper prototypes are probably the fastest to turn around, because they don't require coding. But if there's a working interface or some animatic, the intern could do usability tests based on iterations of that.

For a simpler paper prototype, a more aggressive schedule could be:
  1. Quick study of usability testing background and methods
  2. Review test needs with Design team; plan test #1
  3. Execute test #1
  4. Discuss results with Design Team; plan test #2
  5. Updated prototype; execute test #2
  6. Discuss results with Design Team; plan test #3
  7. Updated prototype; execute test #3
  8. Discuss results with Design Team; plan test #4
  9. Updated prototype; execute test #3
  10. Discuss results with Design Team; plan test #5
  11. Updated prototype; execute test #3
  12. Discuss results with Design Team; plan test #6
  13. Wrap-up

But that may not be realistic, even for a paper prototype. There's still logistical issues, such as finding test volunteers. We'll have to adjust the schedule as we go. I think the thing to remember is we'd target at least 4 and maybe 6 usability tests during the Outreachy cycle.

If you're interested in participating in Outreachy, the anticipated schedule is:

Sept. 10, 2018 Applications and the eligibility check opens. Applicants can see and apply to listed projects.
Oct. 2, 2018 Last date for new mentoring community listings
Oct. 9, 2018 Last date for new internship project listings
Oct. 23, 2018 Application deadline
Oct. 30, 2018 Late application deadline
Nov. 5, 2018 Deadline for mentors to select interns
Nov. 14, 2018 Accepted interns announced
Dec. 4, 2018 to March 4, 2019 Internships period

Tuesday, July 31, 2018

The next step in open data is open source

Governments at all levels are moving to embrace open data, where governments share public data proactively with citizens. Open data can be used, reused, mixed, and shared by anyone.

For example, The US Government has an open data portal that publishes data on various topics, including agriculture, education, energy, finance, and other public data sets. Where I work (Ramsey County, Minn.) we established an open data portal that shares expenses and other public data about the county that users can access in different views.

Through open data, governments become more transparent. We have seen this in several instances. The Oakland Police Department used a 2016 open data study from Stanford University to address racial bias in how officers behave towards African Americans versus Caucasians during routine traffic stops. In 2017, Steve Ballmer launched the USAFacts website that uses open data to reveal how governments spend tax dollars to benefit citizens. Also from 2017, Los Angeles, California’s comprehensive “Clean Streets LA” initiative uses open data to assess and improve the cleanliness of public streets.

Governments at all levels have recognized that open data feeds citizen engagement. By sharing data in a way that encourages citizens to remix and transform open data to provide new insights, governments and citizens move closer together. According to the Open Data Barometer, many municipalities already provide open data for geographic information, transportation, trade, health, and education, with a mix of other open data sets. Those governments that do not yet provide an open data portal are likely working to provide one.

What is the next step beyond open data? After sharing data, what is the next evolution for governments to engage with citizens?

I believe that next step is open source. Where we provide government data sets for anyone to view, adapt, and remix, we need to offer government source code for others to view, adapt, and remix.

While there is a balance to be made in moving to government open source, the default should be to share as much source code as possible. Just as governments found a balance in providing open data, government open source must consider what software can and cannot be shared as open source software. In the same way that some data needs to remain private because it identifies individuals or because it contains certain nonpublic data, some government source code may need to remain “closed source.”

In adopting government open source, we should follow the open data model. The default in government open data is to share as much data as possible, to release public data for public consumption. That should be the same with government open source. In cases where government application development teams write custom software, we should make as much of our source code available to the public as possible.

Some government agencies are already moving to an open source model, and that is good. In August 2016, US Chief Information Officer Tony Scott and US Chief Acquisitions Officer Anne Rung issued instructions for federal departments and agencies “to release at least 20 percent of new custom-developed code as Open Source Software (OSS) for three years.” In support of this directive, the US Government established an open source portal at Code.gov to share government source code under the Creative Commons Zero (CC0) and other open source software licenses. Via the open source portal, users can download open source projects, toolkits, installer profiles, online forms, JavaScript widgets, and other code samples.

The challenge we face in moving to government open source is not technical, but cultural. Many governments have relied on proprietary or “closed source” software for decades. Through the lens of these government IT departments, all software is proprietary. This view often extends to software that is custom-developed by municipalities.

It will take a culture shift for governments to release their source code for public access. But governments faced that same culture shift in moving to open data, and we overcame that cultural inertia. We can do the same with open source.

The benefits to adopting a government open source model are many. Like open data, government open source will provide additional transparency to citizens. Users will be free to investigate the source code, and re-use it for other purposes. Motivated citizen developers may modify the source code to fix bugs or add new features, and contribute those improvements back to the government. This last example is the ideal model, providing a feedback loop of engagement where the government partners with its citizens to improve services.

I believe the next iteration from open data is open source. I encourage government Chief Information Officers at all levels to investigate how software created by government application development teams can be made available to outside users. Use the US open source portal as a model to set goals and measure progress. Finally, establish relationships with partners most likely to engage in government open source, including local universities and businesses.

Through open data, governments became more transparent to citizens. With government open source, Chief Information Officers have an opportunity to lead the next evolution in citizen engagement. Through open source, we can take government transparency to the next level.

Monday, July 30, 2018

What an icon says about you

Icons are an important part of modern computing. We see icons everywhere, from actions on the desktop to navigation in an application to action icons on a website. You can't go long without seeing an icon.

When I first started doing usability testing, I focused on icons. How will users perceive the function exposed by a single button, especially if that button is an icon? Icons can be abstract or representative. For example: in a word processing program, what icon would you click to print a document? The icon you have in mind probably looks something like this: (*wikimedia)

That is a fairly artistic rendering of a printer. But how simple can you make the printer icon, and still recognize it as a printer? (*wikimedia)

This high contrast icon is still recognizable as a printer to many users. Especially when presented on a toolbar with other icons, usually on the left end of the toolbar near the new-file and open-file and save-file icons, you'd probably recognize this as a printer icon. (In this case, certain icons can be learned based on association. In most applications, the "Print" action is usually near the "New File" and "Save File" actions.)

Branding is another form of an icon. You can associate a product or a brand through its icon. You can immediately recognize certain products through their icons, from the Nike "swoosh," the McDonald's "M," the Apple "apple," and the AT&T "Death Star." Open source software has its own recognizable icons, including Tux the Penguin, the GNU gnu, and "Beastie" the BSD Daemon.

Once upon a time, the Netscape "N" was instantly recognizable as the web browser's brand icon. Later, the organization spun off into Mozilla, represented by a less memorable big red dragon head. Finally, we have Firefox, represented by a stylized fox wrapped around a small globe. The fox icon has represented the Firefox brand for years, although now the Firefox organization wants to change the brand icon.

From an article in Venture Beat: "For most people, Firefox refers to a browser, but the company wants the brand to encompass all the various apps and services that the Firefox family of internet products cover," and "The fox with a flaming tail 'doesn't offer enough design tools to represent this entire product family'." The Firefox name will remain, but the branding will change.

This is a tall order. Changing any well-known branding for an organization or a product family is tough. Will users continue to recognize the product as one they have used before? Or will the new branding suggest to users that everything has changed, and the product they've used and known for years has now flipped to something completely different?

Firefox has proposed two possible branding options:


Venture Beat describes the icons as follows: "The first icon is the Firefox masterbrand icon, an umbrella under which all the product lines will live" and "The next [two] lines, in order, are as follows: general purpose browser icons (including Developer Edition and Nightly colors), singularly-focused browser icons (Firefox Focus and Firefox Reality)"and the last five icons are "new applications and services."

First, let me make some guesses about the new applications and services, represented by the five icons in the last two lines. I'm not sure what the first one represents, but I think the second is some kind of photo service, perhaps something like Instagram or maybe a static photo hosting service like Flickr. At least, that's what the green icon suggests to me. The third icon is clearly some kind of password management service, like LastPass or 1Password; the hexagonal "lock" icon is a clearer representation of a password service.

I'm not sure what the last two are. These are too abstract for me to interpret what they mean, but perhaps the last icon is some kind of interaction service, such as a private chat application.

Jumping back the the rest of the list. The first line of icons (the standard web browser, the in-development web browser, and the nightly-build web browser) are still recognizable to me. I don't use the Firefox Developer Edition or the nightly builds, but the different colors in those two icons suggest to me that they are not the standard web browser.

The System 1 icons are more familiar to me, compared to the current Firefox logo. I think if Firefox used the first row icons from System 1, their users would be satisfied that the browser product isn't completely new.

The second row of icons (including Firefox Focus and Firefox Reality) are a mix for me. I'm not sure what Firefox Focus is, based on this image. The Firefox Focus web page describes Focus as a "dedicated privacy browser with tracking protection and content blocking." I don't get that from either the System 1 or System 2 icons. Similarly, the icon for Firefox Reality doesn't suggest what Reality does for me. The Firefox blog describes Reality as a "cross-platform browser for mixed reality." I don't get "augmented reality" from any of the icons in System 1 or System 2.

What about the "parent" logo, the Firefox masterbrand icon at the top? For me, the System 2 icon better represents the "Firefox" family of products. I get the abstract fox icon from System 1, but I think it's too abstract. When I first saw the icon, I assumed it was a red book icon, but the designer had set the alpha channel to be too transparent, so the yellow diamond underneath the book was visible. (And now you probably can't un-see the "book" icon. Sorry.)

But negative space means something, too. As humans, we naturally look for negative space, and read a lot into what shapes we find there. For example, most people see a rightward arrow in the FedEx logo, suggesting speed.

Applying this negative space visual concept to the Firefox logos, my mind naturally tries to find meaning in the negative space of the System 2 icons. I'm less bothered by the negative space of the System 2 masterbrand icon, because the negative space forms a rough circle. But look at the first row of icons. What shape is that? I'm confused.

My recommendation to Firefox is to use the System 2 masterbrand icon, but use the first row of the System 1 icons to represent the web browser. Use the green "Photos" icon (or something like it) and the hexagonal "lock" icon for the Password application.

But go back to the drawing board for Focus and Reality, and the three other extra applications. I'm not sure what those represent, so I think they need more work.

Finally, I encourage Firefox to do a quick usability test. They may have done this already to create this first set of design concepts. Find a bunch of users, and show them the icons in isolation. For one set of testers, don't provide context. "What does this icon mean to you?" Let the user give a reply without knowing what application the icon might be used to represent. For another set of testers, provide some context, but don't indicate which icon is intended for which application. "Which icon represents a web browser?" "Which icon represents a Photos application?" You can probably do this with ten testers in each group, but I'd recommend a larger sample such as twenty testers in each group.

Compare your results. Do the unprompted testers associate the same icon with the same application as the prompted users? If the testers have strong agreement, then you've identified icons that most users will recognize for the purpose you intend.

Thursday, July 5, 2018

First impressions of PureOS

My new Librem 13 arrived yesterday, and it was my first opportunity to play around with PureOS. I thought I'd share a few thoughts here.

First, PureOS uses GNOME for the desktop. And not that it matters much to usability, but they picked a beautiful default desktop wallpaper:


Because it's GNOME, the desktop was immediately familiar to me. I've been a GNOME user for a long time, and I work with GNOME in testing usability of new features. So the GNOME desktop was a definite plus for me.

It's not a stock GNOME, however. PureOS uses a custom theme that doesn't use the same colors as a stock GNOME. GNOME doesn't use color very often, but I noticed this right away in the file manager. Clicking on a navigation item highlights it in a sort of rust color, instead of the usual blue.


Overall, I thought PureOS was okay. It doesn't really stand out in any particular way, and I didn't like a few choices they made. So in the end, it's just okay to me.

However, I did run into a few things that would seem odd to a new user.

What's that file icon?


When I clicked on Activities to bring up the overview, I was confused about what looked like a "file" icon in the dock.


I understood the other icons. The speaker icon is Rhythmbox, my favorite music application. The camera icon is clearly a photo application (GNOME Photos). The blue file cabinet is the GNOME file manager. And the life ring is GNOME's Help system (but I would argue the "ring buoy" icon is not a great association for "help" in 2018; better to use an international circle-"?" help symbol, but I digress).

Two icons seemed confusing. The "globe" icon was a little weird to me, but I quickly realized it probably meant the web browser. (It is.)

But the one that really confused me was the "file" icon, between the camera and the file manager icons. What was a "file" icon doing here? Was it a broken image, representing an icon that should exist but wasn't on the system? I didn't need to click on it right away, so I didn't discover until later that the "file" icon is LibreOffice. I hadn't seen that icon before, even though that's actually the LibreOffice icon. I guess I'm used to the LibreOffice Writer or LibreOffice Calc icons, which is what I launch most of the time anyway.

No updates?


I wanted to install some extra applications, so I launched GNOME Software. And from there, I realized that PureOS didn't have any updates.


Really? Linux gets updates all the time. Even if Purism updated the OS right before shipping my laptop to me, there should have been a few updates in the time FedEx took to deliver the laptop. But maybe Purism is slow to release updates, so this could be expected. It seemed odd to me, though.

Where's the extra software?


Once I was in GNOME Software, I realized the "store" was quite empty. There's not much to choose from.


If this were my first experiment with Linux, I'd probably think Linux didn't have very many applications. They don't even have the Chromium or Firefox web browsers available to install.



But really, there are a ton of applications out there for Linux. It's just the packages that PureOS makes available through GNOME Software seems pretty limited.

The terminal is broken?


Finally, I'll mention the terminal emulator. PureOS doesn't use the standard GNOME Terminal package, but rather the Tilix terminal emulator. It's a fine terminal, except for the error message you see immediately upon launching it:


I wondered why a pre-configured operating system, aimed at the Linux community, would ship with a broken configuration. I clicked on the link shown, and basically the fix is to set Tilix as a login shell, or to do some other setup steps.

Presenting an error message the first time the user runs a program is very poor usability. I haven't run it yet, so I assume the program should be using defaults. Everything should be okay the first time I run the program. I assume things will "just work." Instead, I get an error message. If I were a novice user, this would probably turn me off PureOS.

Overall


In the end, PureOS is a GNOME desktop that runs well. But with a few confusing issues or problems here and there, it doesn't exactly stand out. To me, PureOS is just "okay." It's not bad, but it's not great.

I think my biggest concern as a longtime Linux user is that the distribution doesn't seem to have updates. I'm curious to hear from any PureOS users how often updates are pushed out. I run Fedora Linux, and I get updates pretty regularly. What should I have expected on PureOS?

Tuesday, July 3, 2018

Librem 13: Review

In May, I decided it was finally time to replace my old laptop. Technically, there wasn't anything wrong with my old laptop (Lenovo Thinkpad X1 Carbon, first-gen) but after six years, I thought it was time to replace it.

Of course, I wanted my new laptop to only run Linux. After some searching, I decided on the Librem 13, from Purism. Purism laptops are designed and built for Linux, and I wanted to support a hardware vendor that aimed squarely at the Linux and Free/open source software market.

Unfortunately, I had a few problems with the Librem laptop. The Intel on-board video card "flickered" when I used the internal display, and sometimes would go to "sleep" (not sure it was really in sleep mode or just shut itself off, but when the screen goes black and the laptop is still running, that feels like "sleep" to me). I contacted Purism, and they suggested this was a hardware fault they've seen on some laptops, and they gave me an RMA to return it for repair.

A tech later emailed me to say they couldn't repair the laptop, so they sent me a new one instead. My new Librem 13 arrived today, and it's great!

System information

I've highlighted the ordered specs and the system details so they are easier to compare: memorydisk, and CPU. Here's what I ordered: (copied from my order confirmation)

  • Keyboard: English (US)
  • TPM: Include
  • Memory: 16GB (1x16GB) (+$209.00) $209.00
  • Storage (M.2 SSD): 500GB (NVMe) (+$499.00) $499.00
  • Storage (2.5" SATA 3 SSD): None -$99.00
  • AC Adapter Power Plug: US
  • Wireless: Include Wireless
  • Operating System: PureOS
  • Warranty: 1 Year

I figured I'd max out the memory. I'd like this laptop to last a long time, and memory is a good investment there. Also, I swapped out the standard SATA SSD storage for a 500GB M.2 SSD storage. The prices here reflect those changes.

And the technical details: (*see note)

$ free -h
              total        used        free      shared  buff/cache   available
Mem:            15G        2.5G        5.5G        616M        7.6G         12G
Swap:          7.9G        263M        7.6G

$ cat /proc/cpuinfo |egrep '^processor|^model name'
processor : 0
model name : Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
processor : 1
model name : Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
processor : 2
model name : Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
processor : 3
model name : Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz

$ sudo fdisk -l /dev/nvme0n1
Disk /dev/nvme0n1: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x46f877a1

Device         Boot     Start       End   Sectors   Size Id Type
/dev/nvme0n1p1 *         2048   2099199   2097152     1G 83 Linux
/dev/nvme0n1p2        2099200 940814335 938715136 447.6G 83 Linux
/dev/nvme0n1p3      940815444 976768064  35952621  17.1G 83 Linux

First impressions

Overall
I've been using the new laptop for a few hours now, and I'm happy so far. This is a great system.
Video flicker is fixed
I'm happy to report that the video "flicker" problem is not present on this model! So that seems to have been a hardware fault, and not a driver problem. Very pleased that ended up being a fixable hardware issue.
Wrong key code for backslash and pipe
The keyboard issue is still there. The Purism laptop uses a keyboard that sends the wrong key code for the backslash key (\). The "shift" on this key is the pipe symbol (|). Try running any commands at the Linux command line, and you'll quickly run into a problem where you can't send the output of one program into another program. You need the pipe for that. Or try escaping a character at the command line, or in program code. You need the backslash for that.

This is a known issue on the Librem, but it's easy to fix. You need to run setkeycodes 56 43 to reset the correct key codes for that key system-wide. To make the fix permanent, create a new /etc/rc.d/rc.local file that is executable (I used mode 750, but anything that's executable and owned by root should do) and has these lines:

#!/bin/bash
setkeycodes 56 43
exit 0

This fixes the problem each time the system boots. You don't need to do anything at the user level. Note that I have my Librem connected to an external display, and I'm using an external keyboard and mouse. This key code fix doesn't impact backslash or pipe on my external keyboard, so I'm good there.
Operating system
I did end up re-installing the operating system. When I first booted the Librem, it was using the pre-installed PureOS Linux distribution. I played with it for a while, and actually did some work online with it, then decided I'd rather run the Fedora Linux distribution that I'm used to. I'll post an article later with impressions about PureOS.
*The numbers don't match exactly, and that's expected. Note that free and fdisk display powers-of-ten Gibibytes (GiB), while the specs from Purism display powers-of-two Gigabytes (GB). So 500 GB = 466 GiB and 16 GB = 15 GiB. (see comment)
Update: I'm still having screen flicker problems. It's just not as noticeable because the first screen flicker problem was wide "bands" of what looked like static. This new screen flicker problem basically looks like thin black lines that appear randomly on the screen, but just for a moment. I didn't notice at first because I use an external display as my primary, so my laptop display was just showing the wallpaper (a dark image) and my music player. But yes, Purism confirmed this is another hardware issue, so they are replacing it. They won't have new stock until sometime in August, so I'm using my old laptop until then.

Wednesday, June 27, 2018

A few thoughts on GNOME usability

I recently learned that GNOME developers have proposed moving the application menu off the "top black bar" and into the application window. The proposal is on the GNOME Wiki at App Menu Migration.

The wiki has a few mockups of how the menus would appear. The top-level application menu would be presented as a "hamburger" icon, and any secondary menus would appear as a "three dots" icon.

I have to say that I think this would improve usability. My previous usability testing shows that users prefer menus that are obviously part of the application (think "menu bar"). The application menu in the black top bar doesn't stand out as part of the application experience.

I was never completely a fan of GNOME's decision to put actions behind the application menu. I thought it was sometimes confusing. And in usability testing, many users didn't immediately realize it exposed a menu. Once you click on the application menu, in most applications there's just an option for "Quit" (which is the same as clicking the "x" icon in the upper-right, anyway). But in multi-monitor setups, the application window might be shown on a separate monitor from the display that has the black top bar. So you lose the association between the application menu and the application itself.

Moving the application menu into the application window makes a lot of sense.

My immediate thought is that the icons could be confusing. Instead, I'd recommend a "gear" icon for the top-level application menu, and any secondary menus appear as either the "hamburger" icon or a "three dots" icon. I'd pick the gear icon because most top-level menus are where you find the Preferences. I have a slight preference for the hamburger icon, but prototype usability testing would demonstrate which one was better for the user.

I have two recommendations:

  1. Make it very clear that one button is the top-level menu, and the other button is the secondary menu. For good usability, users want to find the right icon on the first try. They don't want to hunt for the right menu icon. Users need to make an immediate decision which icon to click. And for that, the top-level menu icon needs to be obvious, and the secondary menu icon needs to be visually distinct.
  2. Be consistent in where you place the top-level menu icon and the secondary menu icon. Having learned where to find the menu icons in one GNOME application, users should be able to transfer that knowledge to another GNOME application. My personal preference is to put the top-level menu icon in the left-most position (similar to the clickable program icon menu in GNOME 2) and to put any secondary menu icons in the right-most position. For application windows that use a split view, the secondary menu icon should always be on the right of the "pane" view.

As a quick html mockup of a few title bars shows the top-level menu always the left-most icon, and the secondary menu always the right-most icon:

Application×
Application ×
<Application ×
<Application+ ×
< Applicationz ×
< App name  Detail view+ ×

My html mockup isn't perfect, but I hope you get the idea.

I think the next step is for the GNOME developers to do a few prototype usability tests. Which icon would best carry the meaning of "top-level menu" and "secondary menu"? And where do users think to look for a "top-level menu" button vs a "secondary menu" button"?

Ciarrai did a great job with their prototype usability test of the then-proposed Settings app. Ciarrai and I co-authored an article for Opensource.com called Usability testing for early-stage software prototypes that provides a good model for how to do prototype usability testing. And remember, you only need about five testers to get "good enough" results, if you do iterative usability tests.