The usability study on April 9th at noon was different than the previous three studies for a variety of reasons.

First, we (Clara, David, Jenny, and Sarah) conducted the study at the Wildcat Den on the Downcity campus. Run by off-Campus Student Services, “The Den is a multifunction lounge space for commuting Wildcats.” While it is not clear if the students from previous usability studies live on or off campus (because we do not ask this question), the focused group of test 5 was comprised of students, many of them sophomores, who do not live on campus. This was our first usability study outside of the library.

The second difference is that we tested a rough-draft beta version of the library homepage that offers a side-menu navigation bar instead of tabs along the header. In addition to the header the beta page moves all content above the fold and provides an additional tab dedicated to textbooks.

Please let us know what you think in the comments or let us know directly.

The third difference is that we conducted multiple, simultaneous tests. With four librarians, four laptops, and two sets of headphone/microphones, we were able to conduct four usability tests in half the time by breaking into groups of two. This worked out well because The Den is a small space and its loud and busy during the day. Being able to go in for an hour, conduct four tests, and then leave, was time and effort efficient.

Takeaways

• Students use the “everything” tab to search for everything- not only items that are searchable in our catalogs or databases, but information about library services.
• Students use the tabs along the header (Get Help, Services, About, etc.) but don’t often find what they are looking for there. It was not clear to them what they would find in any of these drop down menus, but they went there anyway. This is good to know when thinking about the redesign: perhaps the header drop down menus are a good place to include information about the library and services, as long as the language is clear and makes sense to the user.
• Students don’t seem to know about the FAQ’s (but when they used the beta site some users intuitively searched the FAQs through the left-hand quick answer search).

Even though we haven’t posted to the blog recently, we have been thinking about, talking about, and developing our fourth usability test.

Welcome to Phase 2 Library Website Incremental Redesign and Guerrilla Usability Study!

In Phase 2, the keywords of the study and redesign are intersectionality and inclusion. Our previous data collection was focused on users within the library. We want to expand the data to capture users who may not feel comfortable coming to the library and increase our inclusivity to marginalized students. We recognize that research shows that the more marginalized the student, the less likely they are to use library services/university services. Yet the more students utilize library and university services, the more successful they will be.

There are two ways we will endeavor toward inclusivity in Phase 2: through a homepage pop-up survey and taking usability test 4 outside the physical walls of the library. We’ve also included a summary of the other redesign projects on which we are working.

Home Page Pop-up Survey

In our effort to be more inclusive (overall, but specifically) to target marginalized students, we will be adding a pop-up survey to the homepage. The survey will collect the demographics of our users and we have added one question to capture first generation status.

We are hoping to learn who some of our users are, what brings them to our website and whether we are providing them what they expect to find.  The survey will launch as soon as next week.

Upcoming Usability Test: Commuter Students in the Wildcat Den

We are finally leaving the library and will be engaging in a usability study in the Wildcat Den with commuter students in mid April. As part of our effort toward inclusivity, we will be doing our best to reach out to students who might not normally come to the library.

We are excited to try something new with usability test 4: test an alternative interface to the homepage. This alternate interface will feature a prominent textbook search and several layout adjustments. We’re looking forward to seeing what we learn and we plan to share our findings in the next blog post.

Change Ideas: Contact Icons

In order to make room for new content we are testing out an alternative contact access point in the top-header on the homepage.

In our tests students were not using the contact icons in the middle of the page and we have hypothesized that this bright orange link will draw their attention. We will measure the success of this by comparing clicks in Google Analytics and testing out contact-related tasks in future usability tests.

Textbook Reserves Tab: work in progress.

The textbook reserves tab project began as a takeaway from one of the earlier usability tests. We found, as we noted in a previous post, that there is a disconnect between the language librarians use for our library resources and the language used by students. One of the terms in question is “course reserves.”

We also observed issues in the functionality of the search (for example, users cannot perform title or author searches). To fix this issue we are going to add a scoped search on the website that will only search items classified as reserve books. Before we can do this, however, we need to edit the MARC records of the textbooks to add an item in the notes field that will allow us to n limit our searches to just textbooks. Clara is working hard on this project and we’re excited to put this into production soon.

Sarah, Clara, David, and Jenny ran the third usability test in two sessions over two days, the first on Wednesday, February 6th and the second test on Friday, the 8th. In this round of the study there were three participants. Test A, on Wednesday was in the library instruction room while Test B, on Friday, was at a table in the silent section section of the library. It is unclear if the change in location affected the participants, although it may have made the testers/librarians less apt to engage in prolonged discussion with the user.  

(from Left to Right) Photo 1. Sarah and David running a usability test with a student on Wednesday. Photo 2. David and library intern, Clara, running a usability test with a student on Friday.

Takeaway #1: “What’s a Libguide?”

From the three usability tests on students that we’ve done so far, we have learned that many don’t know what the terms “LibGuides” and

“research guides” are. We think that students usually view them if told by their professor or through uLearn. It might be a better idea to remove the LibGuide search box at the top right of the page, and replace it with a hyperlink that would bring all users to a page where they can browse all LibGuides/ research guides. For a future usability test, one will be made to test. Lastly, another possibility for the current LibGuide search box is for it to be moved to the header of the page with a drop-down menu, which will provide more space. We will have to monitor how much it gets used.

Takeway #2:  Textbooks vs. Course Reserves

The disconnect between the language that librarians use and that students use and understand extends to the terminology for course reserves. The results of test 3 have made it clear that students are unfamiliar with the term “course reserves." We have come to the conclusion (from the results from the three studies as well as anecdotally, from interactions with students in the library) that not only do we need to refer to “course reserves” as textbook reserves, we also need to change how students access their course reserves. After test 1 we added a yellow “find your textbook in the library” button. This language makes sense to students, however, they are not using the button. Going forward,  we will need to find a new way to provide access to their “textbook” reserves.

We asked some of the same questions from test 2 to get more data, but we also added some new questions and modified others.

Questions

1. Your professor tells you there is a page on the library website for your projects in this class. Have you ever used this? What do you call this page? What do you expect to find on this page?
2. Find an article about a current topic in business.
3. You need to watch a documentary film about the history of educational psychology for one of your classes. How do you find it, using the library website?
4. Your professor for English 1020, Eileen Medeiros, says you can find your textbook, The Norton field guide to writing, in the library. Locate it from the library web page.
5. Take a look at this library website (Cornell). What sticks out to you the most? What do you see? What do you like? How is it different from the JWU library website?
6. Take a look at this website (Wake Forest). What sticks out to you the most? What do you see? What do you like? How is it different from the JWU library website?
7. Do you have any final comments, impressions, thoughts about the JWU library website? What do you like, don't like, find confusing about our website?

Looking Ahead: Usability Testing, Phase 2

We will be taking a few weeks off from the testing and will start in March once the new term begins. During the second phase of the study we would like to learn about how faculty and library staff navigate the website and access resources and services. We are also hoping to usability test with online students as well.

If you are interested in participating in the usability testing or have some thoughts about problem areas of the website, please let us know!

“Above the fold”: Usability Test 2 Discussion

We ran our second usability test on Wednesday, January 30th. Kelly, David, Jenny, and Clara conducted the usability test in the library instruction room. Four students participated in the study.  The tests were each 15 minutes or so. We followed the same roles (reader, note taker, and observer) as with Usability Test 1 and used candy as incentive to participate (it worked this time!).

Questions/Tasks

• Find a libguide (Your professor tells you there is a page on the library website for your projects in this class. Have you ever used this? What do you call this page? What do you expect to find on this page?)
• Find a database without using the search
• Identify where to go for help when stuck
• Find a textbook from the library webpage
• Comment on features and design elements on other library websites (Cornell University Library)
• Provide thoughts/feedback on JWU Library website

Number of Students Observed: 4

Takeaway #1: Keep it Above the Fold

One of the big takeaways from our second week of usability testing is that students do not and often will not scroll. This means that any information posted below the “fold” of the initial screen (anything you have to scroll to see) on a page when you land on that webpage, will most likely not be seen. While learning this was not necessarily surprising it definitely shifted our perspective about the current state of our website and the information that students are seeing as opposed to what they are not seeing when the access the library website.

One of the Week 2 usability test tasks asked users to view the website for the Cornell University Library. We chose this website because we liked the design, the search box and that there is a separate “textbook reserves” search box. Cornell’s page has a different design and functionality than the JWU library website and we wanted to students thought and if they used them differently.

The big difference between our website and Cornell, for students, was that all the information on the Cornell library website is above the fold (our term, not theirs). One student noted “college students do not like to scroll.” After comparing the two sites, another student asked of our library website, “how am I supposed to know to keep scrolling?” This got us thinking: if we moved some of our resources/information “above the fold,” would it get more clicks/have broader access by students?

In the days following the test, we made several changes to the site to increase the amount of content that shows above the fold. There are still several improvements we can make to increase the above-fold content, but for now at least users on a laptop can see the top of the second row page elements (Chat, Contact, Appointment/Study Room).

Before Changes:

After:

Takeaway #2: The Disconnect

While analyzing the test results, we started to suspect that there is a disconnect between how librarians and students viewed library services, and that though we all have a vague shared sense of what a library website (or an academic library in general) is and does, we might differ in the details and in what we prioritize. We first became suspicious when we gave the students their first task (“Your professor tells you there is a page on the library website for your projects in this class”) in an attempt to see how they would find a libguide. No one even considered that this is something that might exist, and the students either searched for articles or a database. We had to repeat the question in different phrasings to get what we wanted, and by the time the students finally landed on a LibGuide we learned that they did not know what a LibGuide (or research guide) was.  We resolved to investigate this further in future tests (watch for out for the Usability Test 3 Blog post!)

Other Small Changes we made as a result of the tests:

• Added “What can we help you find?” to the placeholder text in the search box
• Removed unnecessary page content (title and breadcrumbs on homepage) to make more space
• Updated library logo to include ‘Providence’ (this was necessary after we removed the page title on the homepage)

On 1/22 we ran our first guerilla usability test in the library. We were inspired by a number of sources, including this blog post on the User Cafe at the University of Houston Libraries, the book Don’t Make Me Think, by Steve Krug, and several articles.

Librarians Sarah Naomi Campbell, Jenny Castel, David Meincke, and Intern Clara Mendonca conducted the usability studies in the library instruction room. Tests ran for 5-15 minutes. We recorded the screens and the answers of the participants and reviewed them on the following day. Each of the testers had a role: a recruiter, an observer/note-taker, and the script reader/task assigner.

We offered candy as an incentive, however, none of the participants wanted the candy and it can be assumed that this incentive was not motivation to participate in the test. The first usability test was a success, if success is defined as learning something about the way some (four) students use certain elements of the website and recognizing areas for revision in the test and the script.

Questions (view them here)

What we learned, what we changed

1.  Added "Find your Textbook" button to Books tab in searchbox
   • Users had some trouble completing the course reserve tasks
   • A few of the students looked under the 'Books' tab when asked about textbook, so we added a big orange button (see above)
   • We will be testing this new button and tracking usage as we continue to explore how students find textbooks through the library website
   • Part of this could also be due to terminology--this is something that we will also explore later.
   • ADDED:
2.  Removed lower-middle-right hours box from homepage
   • Users immediately notice the hours in the header therefore this box is redundant.
   • Furthermore, this box takes up prime real estate and pushes important content (research appointments, especially) below the fold.
   • REMOVED:

What we learned about the data collection tool and how to implement it:

•  The social capital and authoritative capital of the recruiter may affect the participant or may provide incentive to participate.
•  Having too many testers may affect the test results (?): too many librarians observing.
•  The questions/tasks should be specific and clearly stated. Asking participants “where do you find help”

Communication

•  As we start making even minor changes to the website, we recognize the importance of communicating with library staff who regularly use the site on their own or with students.
•  Rather than sending multiple emails over the course of this process, though we will still communicate any major changes in this way, we are keeping track of changes in the LibAnswers FAQ module so that staff can access the information at any time.

TLDR:

• The JWULibrary is in the process of evaluating and improving the library website.
• The goal of this project is simple: to match the look of the JWU  website, ensure design creates an inclusive and intuitive user experience (minimal effort)
• We will be documenting our efforts on this blog. We want this to be an inclusive process, and there are a number of ways to get involved.

Background

Since the start of the Fiscal Year in July 2018, our website has been visited over 150,000 times and over 300,000 pages have been viewed.  The website is the access point to many of the library’s services, including study rooms, research appointments, virtual reference assistance, database access, and book searching.

It has been a while since anyone has made any major changes to the JWULibrary website (the last was in 2016, with the addition of a navigational header and a new homepage search box). Since then, the main JWU.edu website has changed, and expectations of what constitutes a modern and usable website have changed. Furthermore, anecdotal testimony suggests that that aspects of the site are difficult to use for some of our users.

Why a Redesign?

In the Fall of 2018, the JWULibrary began a process of creating objectives and action plans that tied into the Guiding Principles of the JWU’s Focus 2020 strategic plan. In order to meet the guiding principle,  “act as a good steward of our resources to support the needs of our students, faculty, and staff,” we were tasked to update the appearance and functionality of the library website.

The core web redesign team (us: David, Jenny, Kelly) formed out of this strategic planning exercise, and over the past several months we have worked on a plan to redesign the library website.

The end goal of this plan is simple: to match the look of the JWU  website, and ensure that our design creates an inclusive and intuitive user experience. To achieve this goal, we will:

• explore other library websites and document successful design elements
• conduct tests on users and our website
• make strategic, evidence-based modifications to our website
• develop a new set of styles that adheres to University standards

Where we are now: Usability Tests

After spending a few months reading and planning, we began running usability tests on JWU students in order to find out how they used our site and what we could do to improve it.

With inspiration from articles and blogposts (notably, “Start with an Hour a Week: Enhancing Usability at Wayne State University Libraries” and Music Library walk-up interviews (U of Houston), we drafted a series of short usability tests that could be conducted in the library on student volunteers.

These usability tests are short: 5-10 minutes per person, with short questions and tasks (eg. ‘Find when the library closes today. How about next week?’) We are still in the early stage of this (we just performed the first series of tests last week), but we have already learned new things and made changes to our site (see the changelog for the full list).

We will write more about these tests in the next post.

Future plans

Short-term: for the near future (at least until May 2019, and likely afterwards) we will be conducting weekly or near-weekly usability tests in various locations and making small, iterative changes to the website. We will let you know whenever we make change, and if it is a major change we will do our best to consult the opinion of stakeholders ahead of time.

Medium-term: Using what we have learned from usability studies, we will develop new pages and tools as needed.  We will also develop new design and style parameters

Longer-term: We will clean up LibGuide and LibAnswer content and create a documented content process. It would be great if we could get a handle on our codebase. Right now we have CSS, JS, and HTML all over the place and it would be nice to have everything in a consistent, sensible location.

How to get involved

Since the website is something we all use every day we would like to make this an inclusive effort.

Here are some ideas as to how you might get involved.

• Be part of group meetings to discuss the Library’s web strategy
• Help with content cleanup (LibGuides and LibAnswers
• Conduct user research
• Peform usability testing (develop questions, ask quesitons, be part of the observation team, or attend follow-up meetings)
• Help with accessibility improvements (testing and design)
• Contribute to list of good websites (and useful design elements)
• Have an idea as to how you might get involved? Let us know!

-Jenny Castel, Kelly Faulkner, David Meincke