Improving Your Website Usability Tests
Email Newsletter
Weekly tips on front-end & UX.
Trusted by 200,000+ folks.
Check the frontend report!
Start with a free demo —
Take a break with a 3D game built on Netlify!
Click here to kickstart your project for free in a matter of minutes.In one of the first usability tests I ever did, I met a lovely old lady who could not use a mouse. She kept lifting it in the air and pointing at the screen, speaking words of encouragement to the cursor. At the end of the test I got absolutely nothing, but she did think I was a “lovely boy” who should meet her granddaughter. Very quickly I learned the value of setting very clear criteria for participant recruitment.
If you’ve ever run a usability test before, you’ll know that it’s not as easy as it looks. Although it’s not rocket science, there are some intricacies that can make a big difference. In this article I share some of the lessons I have learned which should help you avoid your user test turning into a frustrating experience for you or the test participant.
That first year of my career was the most valuable experience I could have had and while I believe that learning through your own mistakes is the best way to learn, we do not always have the luxury to do so. Here are some tips I have learned along the way which should help you quickly improve your usability testing skills and avoid some pitfalls.
Design Your Usability Test Script To Answer Specific Research Questions
When starting a new usability test, don’t assume that all you need to do is pick out the main areas of the website and ask users to complete those tasks. You may well find some useful insights with this approach but don’t be surprised if when you present back your findings you get bombarded with questions from the project stakeholders that you cannot answer.
Key Takeaway
Talk to the people you’ll report back to and ask them what key questions they need the research to answer for them. If you end up with lots of questions, prioritize them and then work out a way to answer them as best as you can. If the question seems too vague or you’re unsure why they are asking it, get clarification. The more you understand the reasons behind the questions, the better equipped you are at answering them by adapting your tasks and questions mid-test.
Give Participants The Confidence To Behave Naturally
When participants turn up to a test, they’re usually not sure what to expect. They’re probably a bit nervous with a camera in their face and someone looking over their shoulder. Don’t be surprised if they look to you for guidance at the beginning. If you’re too controlling at the start of the test, you’ll reinforce to them that they need to get permission from you before they do anything.
Give participants the confidence to behave naturally.
Key Takeaway
Encourage users to show you their natural behavior by starting off your test with a broad task to allow them to go off and explore in any direction they like. I use pre-test questions to uncover a real problem they face within the context of the test and then I let them off the leash to answer it as naturally as possible. For example, I wanted to test an online property law website, so in the first task I asked people to search for a house in the area in which they would like to buy, within a specific budget range. This allowed us to get a realistic view of how they use the web, while also setting the context for the next tasks in the test.
Leave Room For User Freedom To Complete The Task In Their Way
In the early days, I used to set out a task in the test script and as soon as users started to stray off the task I’d reign them in and ask them to try again. Not only was I extremely controlling, sometimes losing rapport with participants, I also denied myself the chance to learn something I hadn’t already expected to find.
Key Takeaway
Always leave room for users to freely roam around the website and go off track a little before bringing them back to the purpose of the task. You may feel you are losing control or that the participant has misunderstood the task, but try to resist temptation for a while longer because it can be fascinating to see where they go and why. Often you’ll uncover some real gems here, so try your best to let it happen. If you need to pull them back on track you can, but not until you’re sure they won’t find their own way back.
Relax, Shut Up And See What Unfolds
It’s easy to be rigid and controlling and only focus on what you need users to do for you. When they do something interesting or unexpected, it is extremely useful to ask them what they are thinking. But do this too early or too often and, like I mentioned above, you could miss out on observing natural behavior.
Let the users’ thought processes flow. (Image by Drew Clemens.)
Key Takeaway
Try your best not to interrupt the flow of the participant’s thought process. The more you interrupt, the less likely they are to have the confidence to complete tasks unaided. If you’re asking them something every 30 seconds, they will keep losing their flow and you won’t see anything like natural behavior. You can always bring them back and ask them what happened afterwards. I see a lot of people new to usability testing make this mistake, so I remind them that it is impossible to ask questions and observe what users are doing at the same time.
Tailor The Tasks To The Participant In Front Of You
I’ve mentioned being too rigid a lot so far. I think its because when you do something new, you like to control the variables and lock down the unknown. But with experience, you learn to release control as you get more confident that you’ll handle anything that comes up.
In my early days I liked to write out the exact scenario I was going to give to users to set the scene for a task. But I soon learned that users simply do not engage as much when I set tasks that do not match with what they would normally do. I remember a time when I asked a 19-year-old guy to imagine he was a mother of three in order to complete a task. Needless to say, he looked at me strangely and didn’t really engage with the task at all before he gave up and said he couldn’t find it.
Key Takeaway
Set the overall task you want users to complete but try to be generic and then tailor the scenario to the participant. While this isn’t always possible, there is huge value in spending a little time at the beginning of the test to learn about who the participant is and their current use of similar products or services. If you can then use this to build a test scenario that fits with a real problem or scenario they would like to solve, you can learn so much more than when someone simply “pretends” to be in a scenario.
Always Include Tasks On Peer Or Competitor Websites
Spending a whole hour on a single website can be boring for you and your participant. But boredom isn’t the only problem. All your findings and observations are based on an isolated case. You have no real understanding of whether that person always goes to the search box, or whether they just did it on your website because they were confused by the navigation options. Just looking at one website doesn’t give you a realistic picture of how people use the web.
Key Takeaway
Make the time in your test plan to look at competitor or peer websites as part of your test. The best way to do this is to ask participants at the start of the test what websites they currently use and ask them to show you. Then you can introduce a competitor or peer website they haven’t used before. You’ll find that you learn much more about their patterns of behavior and why they choose one website over another, and more importantly you’ll learn what works well on other websites and why. This is a great source of inspiration when you need to solve a tricky usability problem on your website.
Don’t Let Them Know Which Website You Are Testing Straight Away
I have made the mistake in the past of making it obvious what website I am testing. Sometimes this is difficult to avoid, but if you can, I would advise it. The main reason is that it can be very hard for anyone to be completely honest about their experience with a website when you work for the company either as an employee or as an agent.
Key Takeaway
If I haven’t been involved in the design of the website prior to the test, I always emphasize my independence. Another trick is to get participants to look at competitor websites and give you honest feedback on them before visiting the actual website you are testing. If you can do this without them knowing which website you are really testing, you have a much higher chance of them offering their honest initial thoughts. Towards the end of the test it is likely to be obvious as you’ve spent most of your time setting tasks on one website, but by then you should have been able to get a good understanding of their honest first impressions.
In Summary
If you want to improve your usability testing technique, there is no substitute for doing more tests. However, as I’ve highlighted here, you can try to be aware of how the design of your tasks and how you interact with the participant can affect the outcome of your research. Designing your test to focus on key research questions and not being too rigid with your tasks is a good starting point. In addition, using competitors as part of the test and encouraging users to behave as naturally as possible can yield better results.
If you want to learn more about how to plan, design and moderate usability tests, I have listed some highly recommended books below:
- A Practical Guide to Usability Testing, Joseph Dumas and Janice Redish (Greenwood Publishing Group: 1993)
- Handbook of Usability Testing: How to Plan, Design and Conduct Effective Tests, Jeffrey Rubin and Dana Chisnell (John Wiley & Sons: 1994)
- Moderating Usability Tests: Principles and Practices for Interacting, Joseph Dumas and Beth Loring (Morgan Kaufmann: 2008)
Finally, if you have any questions please post them in the comments below. I’ll do my best to answer them there, or if warranted, I’ll expand on them further in a future article.
Further Reading
- Prioritizing Devices: Testing And Responsive Web Design
- Noah’s Transition To Mobile Usability Testing
- Where Are The World’s Best Open Device Labs?
- A Guide To Simple And Painless Mobile User Testing
(cp, mrn)
