Sarah, Clara, David, and Jenny ran the third usability test in two sessions over two days, the first on Wednesday, February 6th and the second test on Friday, the 8th. In this round of the study there were three participants. Test A, on Wednesday was in the library instruction room while Test B, on Friday, was at a table in the silent section section of the library. It is unclear if the change in location affected the participants, although it may have made the testers/librarians less apt to engage in prolonged discussion with the user.  

(from Left to Right) Photo 1. Sarah and David running a usability test with a student on Wednesday. Photo 2. David and library intern, Clara, running a usability test with a student on Friday.

Takeaway #1: “What’s a Libguide?”

From the three usability tests on students that we’ve done so far, we have learned that many don’t know what the terms “LibGuides” and

“research guides” are. We think that students usually view them if told by their professor or through uLearn. It might be a better idea to remove the LibGuide search box at the top right of the page, and replace it with a hyperlink that would bring all users to a page where they can browse all LibGuides/ research guides. For a future usability test, one will be made to test. Lastly, another possibility for the current LibGuide search box is for it to be moved to the header of the page with a drop-down menu, which will provide more space. We will have to monitor how much it gets used.

Takeway #2:  Textbooks vs. Course Reserves

The disconnect between the language that librarians use and that students use and understand extends to the terminology for course reserves. The results of test 3 have made it clear that students are unfamiliar with the term “course reserves." We have come to the conclusion (from the results from the three studies as well as anecdotally, from interactions with students in the library) that not only do we need to refer to “course reserves” as textbook reserves, we also need to change how students access their course reserves. After test 1 we added a yellow “find your textbook in the library” button. This language makes sense to students, however, they are not using the button. Going forward,  we will need to find a new way to provide access to their “textbook” reserves.

We asked some of the same questions from test 2 to get more data, but we also added some new questions and modified others.

Questions

1. Your professor tells you there is a page on the library website for your projects in this class. Have you ever used this? What do you call this page? What do you expect to find on this page?
2. Find an article about a current topic in business.
3. You need to watch a documentary film about the history of educational psychology for one of your classes. How do you find it, using the library website?
4. Your professor for English 1020, Eileen Medeiros, says you can find your textbook, The Norton field guide to writing, in the library. Locate it from the library web page.
5. Take a look at this library website (Cornell). What sticks out to you the most? What do you see? What do you like? How is it different from the JWU library website?
6. Take a look at this website (Wake Forest). What sticks out to you the most? What do you see? What do you like? How is it different from the JWU library website?
7. Do you have any final comments, impressions, thoughts about the JWU library website? What do you like, don't like, find confusing about our website?

Looking Ahead: Usability Testing, Phase 2

We will be taking a few weeks off from the testing and will start in March once the new term begins. During the second phase of the study we would like to learn about how faculty and library staff navigate the website and access resources and services. We are also hoping to usability test with online students as well.

If you are interested in participating in the usability testing or have some thoughts about problem areas of the website, please let us know!

“Above the fold”: Usability Test 2 Discussion

We ran our second usability test on Wednesday, January 30th. Kelly, David, Jenny, and Clara conducted the usability test in the library instruction room. Four students participated in the study.  The tests were each 15 minutes or so. We followed the same roles (reader, note taker, and observer) as with Usability Test 1 and used candy as incentive to participate (it worked this time!).

Questions/Tasks

• Find a libguide (Your professor tells you there is a page on the library website for your projects in this class. Have you ever used this? What do you call this page? What do you expect to find on this page?)
• Find a database without using the search
• Identify where to go for help when stuck
• Find a textbook from the library webpage
• Comment on features and design elements on other library websites (Cornell University Library)
• Provide thoughts/feedback on JWU Library website

Number of Students Observed: 4

Takeaway #1: Keep it Above the Fold

One of the big takeaways from our second week of usability testing is that students do not and often will not scroll. This means that any information posted below the “fold” of the initial screen (anything you have to scroll to see) on a page when you land on that webpage, will most likely not be seen. While learning this was not necessarily surprising it definitely shifted our perspective about the current state of our website and the information that students are seeing as opposed to what they are not seeing when the access the library website.

One of the Week 2 usability test tasks asked users to view the website for the Cornell University Library. We chose this website because we liked the design, the search box and that there is a separate “textbook reserves” search box. Cornell’s page has a different design and functionality than the JWU library website and we wanted to students thought and if they used them differently.

The big difference between our website and Cornell, for students, was that all the information on the Cornell library website is above the fold (our term, not theirs). One student noted “college students do not like to scroll.” After comparing the two sites, another student asked of our library website, “how am I supposed to know to keep scrolling?” This got us thinking: if we moved some of our resources/information “above the fold,” would it get more clicks/have broader access by students?

In the days following the test, we made several changes to the site to increase the amount of content that shows above the fold. There are still several improvements we can make to increase the above-fold content, but for now at least users on a laptop can see the top of the second row page elements (Chat, Contact, Appointment/Study Room).

Before Changes:

After:

Takeaway #2: The Disconnect

While analyzing the test results, we started to suspect that there is a disconnect between how librarians and students viewed library services, and that though we all have a vague shared sense of what a library website (or an academic library in general) is and does, we might differ in the details and in what we prioritize. We first became suspicious when we gave the students their first task (“Your professor tells you there is a page on the library website for your projects in this class”) in an attempt to see how they would find a libguide. No one even considered that this is something that might exist, and the students either searched for articles or a database. We had to repeat the question in different phrasings to get what we wanted, and by the time the students finally landed on a LibGuide we learned that they did not know what a LibGuide (or research guide) was.  We resolved to investigate this further in future tests (watch for out for the Usability Test 3 Blog post!)

Other Small Changes we made as a result of the tests:

• Added “What can we help you find?” to the placeholder text in the search box
• Removed unnecessary page content (title and breadcrumbs on homepage) to make more space
• Updated library logo to include ‘Providence’ (this was necessary after we removed the page title on the homepage)

On 1/22 we ran our first guerilla usability test in the library. We were inspired by a number of sources, including this blog post on the User Cafe at the University of Houston Libraries, the book Don’t Make Me Think, by Steve Krug, and several articles.

Librarians Sarah Naomi Campbell, Jenny Castel, David Meincke, and Intern Clara Mendonca conducted the usability studies in the library instruction room. Tests ran for 5-15 minutes. We recorded the screens and the answers of the participants and reviewed them on the following day. Each of the testers had a role: a recruiter, an observer/note-taker, and the script reader/task assigner.

We offered candy as an incentive, however, none of the participants wanted the candy and it can be assumed that this incentive was not motivation to participate in the test. The first usability test was a success, if success is defined as learning something about the way some (four) students use certain elements of the website and recognizing areas for revision in the test and the script.

Questions (view them here)

What we learned, what we changed

1.  Added "Find your Textbook" button to Books tab in searchbox
   • Users had some trouble completing the course reserve tasks
   • A few of the students looked under the 'Books' tab when asked about textbook, so we added a big orange button (see above)
   • We will be testing this new button and tracking usage as we continue to explore how students find textbooks through the library website
   • Part of this could also be due to terminology--this is something that we will also explore later.
   • ADDED:
2.  Removed lower-middle-right hours box from homepage
   • Users immediately notice the hours in the header therefore this box is redundant.
   • Furthermore, this box takes up prime real estate and pushes important content (research appointments, especially) below the fold.
   • REMOVED:

What we learned about the data collection tool and how to implement it:

•  The social capital and authoritative capital of the recruiter may affect the participant or may provide incentive to participate.
•  Having too many testers may affect the test results (?): too many librarians observing.
•  The questions/tasks should be specific and clearly stated. Asking participants “where do you find help”

Communication

•  As we start making even minor changes to the website, we recognize the importance of communicating with library staff who regularly use the site on their own or with students.
•  Rather than sending multiple emails over the course of this process, though we will still communicate any major changes in this way, we are keeping track of changes in the LibAnswers FAQ module so that staff can access the information at any time.