Friday, February 12, 2016

Feedback on teaching open source usability

In Fall semester, I taught an online class on the usability of open source software (CSCI 4609 Processes, Programming, and Languages: Usability of Open Source Software). CSCI 4609 was an elective in the computer science program at the University of Minnesota Morris, where I was the Director of Information Technology. (I have since moved on to a new CIO position in St. Paul, Minnesota.) The class was taught entirely online.

I was pleased that ten students signed up for the elective. This may seem small, but it is a significant number for a campus of some 1,900 students and a small computer science department. The same number of students also signed up for other electives that semester, including a course on databases. I organized the class similarly to the usability projects I mentor for Outreachy. Over thirteen weeks, students learned about open source software and usability testing. Most weeks included two assignments: summarizing several assigned articles, and exercising their knowledge of that week's topic. Later in the semester, students moderated two in-person usability tests; the second was their final project.

At the end of each semester, students responded to a course evaluation, called the Student Rating of Teaching. The evaluation is totally anonymous. I don't know which students made which comments, or indeed which students chose to respond to the survey.

This morning, I received my course feedback. I was pleased to see students rated my first course so highly! On a six-point scale, students rated CSCI 4609 Usability of Open Source Software at 5.67 (very good!) or above in every category.

What went well? One of the fill-in-the-blank questions asks students how the instructor most helped them to learn the material. Students thought my responses and probing questions in the weekly discussion were very good. One student commented that my feedback really got the discussions going. Students also thought the links provided in the assignments and in my discussion comments gave them additional insight. I'm glad my teaching style was well received!

For me, the key question in the survey asks what suggestions the students would make to improve the course. I consider feedback a "gift" and I like to hear how I can make the course better the next time I teach it. The responses were not surprising:

More time for the final project
For the final project, I asked students to perform their own usability test on an open source software project they chose. We walked our way up the final project by defining the possible users for their chosen program, documenting Personas and Scenarios, and crafting Scenario Tasks that formed their usability test. Over two weeks, students moderated their own usability test with at least five participants.

But I think the next time I teach this class, I will allow more time for the final project. I thought five testers over two weeks was enough time, but I didn't account for the Thanksgiving break, so students effectively had a week and a half, about eight class days. In addition, this is a busy time of the semester in every other class, and students felt pressured by time. Starting a week earlier and allowing three weeks would be better. So I'll plan that for the next time I teach the class.
Spread out the discussion and comments throughout the week
Most weeks had two assignments: Read several articles I set for them, and summarize them. And exercising their new knowledge by applying it. For example, in one week, we learned about Personas. I assigned several online articles, and asked students to answer the question "What is a Persona?" and identify a few distinguishing features of Personas that make them useful to usability testing. To exercise their knowledge, I asked students to write two or three Personas for their favorite open source software project. To keep the discussion going, I asked students to comment on at least three other classmates' posts.

But I made a mistake in setting these assignments. Each assignment was due at the end of the week. I assumed students would start their work right away, maybe read the articles on days one and two, and start posting on days three and four, leaving plenty of time to reply to each others' comments. But in the real world, students are busy. While one or two students would post right away, most waited until the end of the week to read the articles and post their work. This left very little time for students to comment on each others' posts.

When I teach this course again, I will do this differently. I'll space out the assignments throughout the week. For example, read the articles and post your summary by mid-week, then post your exercise by the end of the week. I might even set separate due dates for the initial post and the comments. This would provide a better structure for students to reply to posts and keep the discussion going: your summary might be due by day two, comments by day three, your exercise by day five, comments by day seven.
I was really excited to teach this online course! The topic of usability in open source software is often overlooked. I enjoyed teaching the class, and I look forward to the opportunity teach it again!
image: University of Minnesota Morris

No comments:

Post a Comment