Usability testing, also known as UX testing, is a popular type of user testing, and it’s considered the best way to evaluate the usability of a product by testing it with real people. The valuable insights derived from usability testing aids design changes in ensuring the UX elements are user-friendly and provides a pleasant user experience.
Oftentimes, we focus so much on the usability testing itself that we often overlook the preparation and other processes. These six common mistakes will affect the user testing negatively and prevent us from getting the best out of the usability testing sessions.
Let’s look at these 6 common mistakes individually and how we can avoid making these mistakes:
1. Recruiting The Wrong Testers
You may be wondering, how is it possible to recruit wrong testers? A tester is a tester, right?
Well, one of the main advantages of usability testing is the ability to test with real users. This means that the testers we recruit should represent your actual target users, instead of a willing friend or helpful relative that has time to spare.
By recruiting the wrong tester, it compromises the validity because their mental models and insights will differ to testers that really represent your target users.
It can be challenging to find the right testers, and so it’s tempting to just settle for testing with colleagues, friends and family that aren’t your target users. This will affect the results of the study, making it biased and invalid.
Got a lot of questions when it comes to respondent recruitment? Answered: Top Questions For Research Respondent Recruitment
Solution:
- Write down a clear and concise recruitment criteria that properly represents your target users
- Create a screener to filter and verify the eligibility of potential testers
- Ensure that all the recruited testers match the profile of your target users
- Build a group that reflects your target users. For example, if 80% of your website visitors are female, recruiting 50/50 by gender isn’t representative.
2. Not Having A Clear & Thought-Out Testing Plan
Failing to have a clear and thought-out test plan affects the entire usability testing, impacting the test results and findings. In addition, the time, effort and money spent on the usability testing won’t yield the best return on investment if the testing goals and objectives aren’t properly defined.
Solution:
- Identify testing goals and objectives at the very start
- Revisit these goals and objectives periodically throughout the process to ensure that the testing is on track
- Evaluate the test results according to the goals and objectives that have been set
3. Testing Too Late
There is indeed a thing as testing too late, testing at the final stages uncovers usability issues that could have been uncovered and solved much earlier.
You may be thinking that it’s impossible to test any earlier because it hasn’t been developed yet. Well, usability testing can be done using mock-ups or prototypes and still be able to derive sufficient insights for improving the UX elements in order to help the final product be the best version it could possibly be.
The later the testing, the more costly things become. This is because at the late stage of the development, any changes would involve changes on design and at code level. Therefore a lot more re-work is needed to make a change. In comparison, if you test and make changes at the very early stage (such as during the clickable prototype stage), the cost of change is much lower.
Solution:
Test as early as possible, even if it’s just a black and white prototype. The initial findings will help to guide the rest of the design process.
4. Not Testing Frequently
Once is enough but it really isn’t as good as compared to testing frequently. You will find more insightful information the more you test. The more you test, the more improvements you can make, thus creating an even better user experience.
Not testing frequently only serves to limit the findings you could make, and so as a consequence, limits the improvements you can make to the user experience.
Solution:
Ensure you test iteratively by conducting multiple rounds of testing throughout the project lifecycle. At the early stages, you can test with low-fidelity prototypes, then with high-fidelity prototypes, and lastly with the finished product.
5. Overloading & Misleading the Testers
There is a thing as too much information when it comes to the task scenarios, there is a possibility of misunderstandings or spoon-feeding the testers instead of seeing how they would act in a normal scenario.
The words we use in the scenarios and test script will impact testers’ actions, and overloading the testers with too much information will keep them from focusing on the actual test.
Solution:
- Don’t give the testers step by step instructions, it defeats the purpose of usability testing
- Provide the testers with just enough information, giving them a background and scenario for them to intuitively complete the user journey naturally
- It’s beneficial to conduct a pilot test to work out the test script, ironing out any overloading or misleading information
- Set clear task scenarios that don’t spoon-feed the testers, For example, “Imagine you’re looking to buy a pair of shoes for yourself for less than RM 300. Can you show me how you would do this on this website?”
6. Testing Only to Confirm Your Ideas
Confirmation bias happens when there is a direct influence to confirm ideas, creating a tunnel vision to dismiss any evidence that doesn’t support those ideas and only looking for evidence, however little, that supports it.
For example, your idea is that the button to checkout works perfectly, and you only collect evidence that supports your theory and idea, ignoring all the other testers that faced an issue with that button.
This can often happen when UX designers influence the usability testing, which is why it’s important to have a middle-person, the user researcher. The fact is, the opinions of the UX designer won’t always match the opinions of the users.
The problem with testing only to confirm your ideas and dismissing all the other valuable insights, is that it creates a constraint from getting reliable results. Not looking at the findings as a whole with confirmation bias will ruin the entire UX effort.
Solution:
- Stay neutral the best you can, it may be a good idea to take a step back from the usability testing process to avoid compromising the results
- Share your assumptions with the others, it’s useful to have various perspectives
- Don’t rely on your own opinion when evaluating the findings, look at the actual results without dismissing anything that doesn’t support your ideas
- Don’t influence the testers’ actions in any way, whether it’s through a conversation, or in the test-script or follow-up questions
Related Articles for Usability Testing:
Usability Testing 101: What, Why, How?
Remote Usability Testing 101: What, Why, How?
Best Practices for Remote Usability Testing
