Monday, February 29, 2016

Internships in usability testing

Are you interested in free and open source software? Do you use GNOME? Then consider applying for an internship with Outreachy!

GNOME and other projects are offering paid, mentored, remote internships to people from groups underrepresented in free and open source software as part of the Outreachy project. This round is open internationally to women (cis and trans), trans men, and genderqueer people. Outreachy is also open in the U.S. to participants with Black/African American, Hispanic/Latin, American Indian, Alaska Native, Native Hawaiian, or Pacific Islander backgrounds.

I've mentored two previous cycles for Outreachy, both in usability testing. I have committed to mentoring again in this cycle. You can see mine and other projects at GNOME's Outreachy project ideas on the wiki. Internship dates are May 23 to August 23. Learn more and apply by March 22.
Usability testing (mentor: Jim Hall)

Benefits: Usability is important in any user interface. Usability focuses on the user and how they expect to use the software. The general rule about usability is that real people use programs to be productive, and they are busy people who are trying to get things done. GNOME needs to be easy to use if we expect others to want to use it.

Importance: Through usability testing, we can make sure everyone can use GNOME.

Requirements: Pretty flexible. It will help if you have some experience with user interfaces, even some coursework in this topic.

Mentorship: I have been very involved in the usability of open source software since I made it the focus for my Master's degree in 2014. I have mentored two cycles of Usability testing in GNOME, and taught an online class on the usability of open source software (CSCI 4609 Processes, Programming, and Languages: Usability of Open Source Software). I also write occasional articles in Linux Journal and other places about usability in open source software.

Project team: The project team will be you and me, pretty much as a two-person team. I find it works best if we follow a course outline: every week, I'll ask you to do some research and write about it on your blog. Over the course of the internship, you'll work your way up to a formal usability test of GNOME, and conclude with analysis and recommendations.

Note: You can learn more about usability testing methods by following my blog at Open Source Software and Usability. You can also search my blog for previous usability testing in GNOME, to get an idea for the work expected.
Please re-post to help spread the word!
image: Outreachy

Friday, February 12, 2016

Feedback on teaching open source usability

In Fall semester, I taught an online class on the usability of open source software (CSCI 4609 Processes, Programming, and Languages: Usability of Open Source Software). CSCI 4609 was an elective in the computer science program at the University of Minnesota Morris, where I was the Director of Information Technology. (I have since moved on to a new CIO position in St. Paul, Minnesota.) The class was taught entirely online.

I was pleased that ten students signed up for the elective. This may seem small, but it is a significant number for a campus of some 1,900 students and a small computer science department. The same number of students also signed up for other electives that semester, including a course on databases. I organized the class similarly to the usability projects I mentor for Outreachy. Over thirteen weeks, students learned about open source software and usability testing. Most weeks included two assignments: summarizing several assigned articles, and exercising their knowledge of that week's topic. Later in the semester, students moderated two in-person usability tests; the second was their final project.

At the end of each semester, students responded to a course evaluation, called the Student Rating of Teaching. The evaluation is totally anonymous. I don't know which students made which comments, or indeed which students chose to respond to the survey.

This morning, I received my course feedback. I was pleased to see students rated my first course so highly! On a six-point scale, students rated CSCI 4609 Usability of Open Source Software at 5.67 (very good!) or above in every category.

What went well? One of the fill-in-the-blank questions asks students how the instructor most helped them to learn the material. Students thought my responses and probing questions in the weekly discussion were very good. One student commented that my feedback really got the discussions going. Students also thought the links provided in the assignments and in my discussion comments gave them additional insight. I'm glad my teaching style was well received!

For me, the key question in the survey asks what suggestions the students would make to improve the course. I consider feedback a "gift" and I like to hear how I can make the course better the next time I teach it. The responses were not surprising:

More time for the final project
For the final project, I asked students to perform their own usability test on an open source software project they chose. We walked our way up the final project by defining the possible users for their chosen program, documenting Personas and Scenarios, and crafting Scenario Tasks that formed their usability test. Over two weeks, students moderated their own usability test with at least five participants.

But I think the next time I teach this class, I will allow more time for the final project. I thought five testers over two weeks was enough time, but I didn't account for the Thanksgiving break, so students effectively had a week and a half, about eight class days. In addition, this is a busy time of the semester in every other class, and students felt pressured by time. Starting a week earlier and allowing three weeks would be better. So I'll plan that for the next time I teach the class.
Spread out the discussion and comments throughout the week
Most weeks had two assignments: Read several articles I set for them, and summarize them. And exercising their new knowledge by applying it. For example, in one week, we learned about Personas. I assigned several online articles, and asked students to answer the question "What is a Persona?" and identify a few distinguishing features of Personas that make them useful to usability testing. To exercise their knowledge, I asked students to write two or three Personas for their favorite open source software project. To keep the discussion going, I asked students to comment on at least three other classmates' posts.

But I made a mistake in setting these assignments. Each assignment was due at the end of the week. I assumed students would start their work right away, maybe read the articles on days one and two, and start posting on days three and four, leaving plenty of time to reply to each others' comments. But in the real world, students are busy. While one or two students would post right away, most waited until the end of the week to read the articles and post their work. This left very little time for students to comment on each others' posts.

When I teach this course again, I will do this differently. I'll space out the assignments throughout the week. For example, read the articles and post your summary by mid-week, then post your exercise by the end of the week. I might even set separate due dates for the initial post and the comments. This would provide a better structure for students to reply to posts and keep the discussion going: your summary might be due by day two, comments by day three, your exercise by day five, comments by day seven.
I was really excited to teach this online course! The topic of usability in open source software is often overlooked. I enjoyed teaching the class, and I look forward to the opportunity teach it again!
image: University of Minnesota Morris