Note: The below test results require understanding about the UIs in question. Follow the links in the beginning to have an idea what the UIs in question are like.

I had eight test subjects during three days usability tests last week. The tests took about 15 minutes each, but gave plenty of data. The main conclusions:

  • Uploading images to the rich text editor in Moodle 2.0 has too many steps and they are well hidden. All 8 of the users really struggled in one or more of the steps (different users in different ones). If it weren’t for a test situation where users typically try more (since they assume the task is possible to do and there is social pressure), even more of them would have likely failed the task.
  • The old ‘forgotten password’ form failed or caused struggles in 4 out of 5 tests. The new form caused no confusion in any of the three tests. (I intended to have an equal number of tests for both. This was a mistake on my part.)

This is discussed in the tracker item MDL-16597, along with proposed solutions (Updated September 2nd).

File Upload in the Rich Text Editor

In the foreseen Moodle 2.0, it is possible to upload images whenever there is a rich text field. Except for the [Choose…] button in the Insert/Edit Image dialog, this diagram/mockup is a very close image of the functionality in Moodle 2.0 HEAD I tested, although there were only three sections int

  • Getting from the text editor to the dialog where you can select which image to upload takes five clicks. Users got lost in all four steps except the last one (pressing the “Browse…” button), most of them in several steps.
    • 1. Get to the add image dialog (4 users found toolbar button, 1 found the item in the right click menu for opening the dialog)
      • Failure: 2 subjects (needed my help to proceed)
      • Struggles: 3 subjects (searched how to do it for several minutes, trying clipboard and drag&drop etc. but continued on their own)
      • Passed this step without struggles: 3 subjects
    • 2. Click the ‘browse’ button (URL is a technical abbreviation and got many users lost; some complained that they do not know what it means; one user complained that URLs have nothing to do with uploading an image from the local computer and was confused due to that. Some users wrote the image name in the description/title field or both; they did not understand the difference between the fields)
      • Failure: 2 (1 needed my help to proceed; 1 gave up at this point)
      • Struggle: 2
      • Passed this step: 4
    • 3. Click “Upload a file” (after this clicking “Browse…” and finding desktop where the file was, was quick)
      • Failures: 0
      • Struggles: 4 (In general, users assumed ‘Local files’ to mean the local computer and first thought they should look there. Thus they  were not looking for upload anymore – apparently they assumed they were uploading already.)
      • Passed this step: 3
    • 4. Press “Browse…”
    • Passed this step: 7

Notes:

  • One or two of the users did not recognize the rectangle under the file upload field as a button so got stuck for a moment there.
  • One user did not understand they still had to click the Insert button in the first dialog to actually get the image in the document.
  • Users ignored “Current files” section, some of them wondered what that is

Forgotten password form

The old form was confirmed misleading since it made 80% of the subjects fill both of the fields. Some of the subjects still did not understand what to do after the form gave the error message to fill one field or the other – apparently the visual (erroneous) message given by the form was a stronger clue to them. What the original forum thread reported was clearly right. I am really surprised such an issue has not been spotted before.

When it was too late, I noticed the WordPress people have designed it even better, though than I did. How bitter: I did not believe it when someone told me having a single field might be a better solution. Now that I see it I think that might have worked better. (Especially due to the formslib bug we have that makes it impossible to have two forms on the same page and thus probably makes the current solution not accessible). Nonetheless, since we now have promising usability test results for my design and none for that of WordPress, I still recommend keeping my design (patch) for now.

Other notes

  • TinyMCE and other parts of the file uploading were in English, although the browser and rest of Moodle were in Finnish since all the test subjects were Finnish. This may have slowed users down but all the test subjects demonstrated during the tests that they understood the English they encountered.
  • All test subjects were my friends and in theory this might have introduced a bias. However, as they were unfamiliar with the UI in question and I did not help them during their test taking, the issues they encountered seemed genuine. This can be disputed though and I welcome discussion about this style of low-fidelity testing.
  • The test subjects were Finnish young people between ages 20 and 30. Three males, five females. Occupations: software development basic degree graduate, musician, social worker student, industrial engineering student, unemployed, interactive technology student, college teacher, kindergarten teacher

The videos taken (screen image and recorded voice in Finnish) are available on request.

I hope to explain how I did this round of testing and which were the parts where I made it easier for myself. There will hopefully be a blog posting or a video, to show the community how little work this can be.

The test preparatory talks and setting were roughly the same as in last summer’s Quiz UI tests, though less formal.

The test tasks

1. Käynnistä internet-selain.

Start up the internet browser.

2. Kirjaudu sisään Moodleen. Tunnukset löydät näytön alareunaan kiinnitetystä lapusta.

Sign in to Moodle. You can find the credentials in the piece of paper attached to the lower border of the screen.

[The username and password are invalid in reality]

3. Näyttää siltä, että vaikka sinulla on käyttäjätunnus ja salasana, toinen niistä on virheellinen. Ota selvää, mitkä oikea käyttäjätunnus/salasana ovat, jotta voit taas kirjautua sisään.

It seems, that although you have a username and password, either of them is incorrect. Find out, which are the right username/password, so that you can sign in again.

4. Testin järjestäjä avasi eteesi dokumentin. Lisää tietokoneen työpöydällä oleva kuva tähän dokumenttiin.

The organizer of the test opened a document in front of you. Add the image on the desktop of the computer into this document.

Proceed