Reference Statistics and Input Form Design

At my library, as in probably every academic library, we collect statistics about pretty much everything that happens inside and even outside the library. One set of statistics we collect is about the questions we get at the reference desk, now called the Research Assistance Desk. We collect data on the type of question it was, how difficult it was, what time of day it was collected, and a general description of the question (which is optional, and no one I know enters anything there). There is a statistics input form for these reference desk statistics. It’s a survey designed in SurveyMonkey. I have noticed that the form is not intuitive. The design of the form and the questions are unclear. I have some ideas for how to fix it, and I’ll be meeting with our Head of Learning, Research, and Engagement next week to discuss some ways to make it better.

Currently, the page is set up with three sections. The first section asks for information on the supposed difficulty of the question. It is a 5-point scale with the following degrees, exactly as they are on the form:

  1. (directional)
  2. (easy, known item)
  3. (use catalog/databases)
  4. (use multiple databases, hard)
  5. (very difficult; referred to SL)

Under each of these options, we can choose “Desk/Phone,” “Chat,” or “Email” to indicate the way the question came in. These are set up so you can choose only one “difficulty” setting, but you can choose multiple ways.

Then the second section is about the day and time the question came in. The days of the week are in a column on the left-hand side of the screen, and various time blocks are set up across the columns. The time blocks are 3-hour blocks starting at 9 am and ending at 12 am. These are set up with check boxes, so I could choose as many day/time slots as I want.

The third section asked “What was the question about? (optional).” There is a free text input box, which nobody actually fills out.

Here are some of the issues I find with it and my recommendations:

  1. The scale for the first section is not even. I think we’re less interested in the level of difficulty of the question than we are in what we had to do to answer it. For example, if I choose #4, then I am saying it is a 4/5 difficulty level, but what they really care about was that I used multiple databases to answer the question. So we should remove the numbers, which imply a scale of difficulty, and simply put choices for how we answered the question. The first choice could be “directional,” the second one, “known item,” the third one, “used catalog or database,” etc.
  2. After that initial question, we could ask which way the question came in. We can have choices for walk-in, phone call, email, and chat message. It should be separated from the first question.
  3. Third, we should redesign the day/time section to look more like a calendar, which is the way most people think in their head. I think that way, at least. I see a calendar with days across the top, and times down the side. It would essentially be an inverted version of what we currently have. This will make it easier to locate the appropriate day and time. Let’s also put Sunday first and end with Saturday since that’s the way most calendars read.
  4. If we want to simplify #3 even more, why don’t we eliminate the day and time question altogether and use the statistics software’s built in time stamp? It’s in every survey software, why not use it?
  5. If we do keep the day and time question, but invert the table, we should break up the time into smaller increments than the current 3 hours. With 3-hour blocks, it’s hard to determine which block you fall in, especially when the reference desk shift times don’t line up with the blocks on the form. We should put 1-hour blocks on there and let the person pick their appropriate time. The way it is not, if I answer a question at 2 pm, then I have to think OK, which time block is 2 pm in? OK, it’s in the 12 – 3 time block. But if I had 1-hour blocks, I’d simple go straight to the 2 – 3 time block. We could also stop at 9 pm since we never offer research assistance after 9 pm anyway.
  6. My last suggestion is to either make the question description required or drop it altogether. It’s just a wasted question if we don’t answer it. It would provide valuable data, but we have to make it a required question.

In sum, let’s separate the “level of difficulty” (which is actually just what we had to do to answer the question) from the way the question came in. Let’s invert the day/time table so that days of the week are across the top starting with Sunday, and 1-hour time blocks are down the left side starting at 9 am and ending at 9 pm.

I think with these changes, the input form would be much simpler and much more intuitive for those of us working the desk. The simpler the better. You’ll get better data that way. You’ll get a better response rate. In all, it will just be better.


You make a good point, but...

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s