Remote Usability Testing has become more popular in 2020 due to the surge of remote work and remote user research. Usability Testing, also known as user testing or UX testing is a research method to evaluate the usability of a product by testing it with real users.
Before we go into the best practices for remote usability testing, let’s take a brief look at the different types of usability testing. Usability Testing can be categorised into 3 domains:
Before we go into the best practices for remote usability testing, let’s take a brief look at the different types of usability testing. Usability Testing can be categorised into 3 domains:
1. Moderated VS Unmoderated
Moderated usability testing requires a trained moderator to moderate the testing session, the moderator will guide and observe the participant, take note of their feedback, and ask follow-up questions in ensuring the testing process is smooth and effective. Moderated usability testing is pricier compared to unmoderated due to the additional time, effort and resources required, but the price pays off as it returns with a more in-depth understanding of the usability issues at the hand of a trained moderator. Unmoderated usability testing is conducted without any moderators to guide the participant, this type of usability testing provides less feedback compared to moderated usability testing. Participants tend to give up on tasks when it comes to complex workflows, especially when there isn’t a moderator present to facilitate the process when the participant is stuck. However, unmoderated usability testing works well when it comes to observing and measuring behaviour patterns.2. Explorative VS Assessment VS Comparative
Explorative testing is a form of open-ended testing that is conducted at the initial stages of development. Explorative testing requires participants to share their thoughts and opinions on the concept and its usability, this helps in the understanding of gaps and potential new features in the market. Assessment testing is used in evaluating user satisfaction and the product’s usability in general, focusing on assessing how well the participants are able to use the product. Comparative testing aims to compare two products or websites, like A/B testing where participants analyze and compare both instances and choose their preferred solution.3. In-Person VS Remote
In-person testing is conducted face-to-face, it’s more fruitful compared to remote testing because additional data points such as body language can be observed. Whereas remote testing can be conducted over the internet or over the phone, and and faster compared to in-person testing. This is because participants do not need to travel for the meeting and can connect as long as they have a stable internet connection and technology devices (you can read about the best practices of participant recruitment in South East Asia). As a result, certain costs such as venue rental and logistic coordination can be saved when running remote testing. However, fewer data points can be obtained in remote-testing as the participants’ body language isn’t easily observed.What is Remote Usability Testing?
Remote Usability Testing is a form of user research conducted in the participants’ natural environment where the screen is shared with the participant or user research software is used in conducting the usability tests. This form of testing is especially beneficial for tight timelines, smaller budgets or hard-to-find participants. Instead of travelling into a UX lab to participate in the testing session in-person, the participants can participate in the user testing session from their home, their office or any other suitable location. The observers can also watch the test sessions remotely.Moderated VS Unmoderated Remote Usability Testing
Moderated usability testing involves a moderator to be present during the test session to guide the participant. The moderator will watch the participant perform the tasks in real-time using screen-sharing softwares, and ask follow-up questions. Unmoderated usability testing doesn’t require a moderator to be present during the test session, so there aren’t any follow-up questions for the participant and the results of the testing process are in the form of recorded videos.Tools for Remote Usability Testing
Zoom
Zoom has become one of the most popular buzzwords in 2020, chances are that you have used it or have at least heard of it. It’s a video conferencing software that has nifty features in enabling recording and live screen sharing, this is especially helpful for moderated remote testing.GoToMeeting
GoToMeeting is a popular web conferencing tool amongst remote workers however it can be quite difficult to set up and use, and it’s considered more expensive compared to other tools.Lookback
Lookback was built for user research and can be used for self-moderated tests, recordings are also available in capturing the user’s screen and webcam.Userzoom
Userzoom provides different methods for remote testing, and even provides recruitment services for the testing process. For a detailed breakdown of pros and cons of different online tools/platforms and for more information, read this article: The Silver Lining of Covid-19: Empowering Remote User Experience ResearchBenefits of Remote Usability Testing
- It’s a preferred and safer method of conducting user research due to the global pandemic
- There’s no need to use or hire a UX lab
- It’s less expensive compared to in-person testing
- Remote usability testing provides a bigger capacity to test with more participants compared to in-person testing
- There are no geographical limitations, diverse groups of participants can be easily accommodated
- Finding participants for remote usability testing can be less time-consuming and less expensive than sourcing participants for in-person testing
- The Hawthorne effect can be reduced since participants are taking the tests in their natural environment instead of a UX lab
Disadvantages of Remote Usability Testing
- For unmoderated remote usability testing, participants may abandon the test or may not pay proper attention without the presence of a moderator.
- Moderated remote usability testing may still cause the Hawthorne or observer effect since they are being observed by the moderator.
- Moderated remote usability testing can be as time-consuming as in-person testing due to the required presence of a moderator.
- There may be a lack of accuracy in unmoderated remote usability testing as the participants have only the script and tasks to guide them instead of having a moderator present to facilitate.
- For unmoderated usability testing, no immediate follow-up questions can be asked.
- Remote usability testing poses a risk where participants take part just to make a quick buck instead of providing valuable and reliable information.
Tips for Remote Usability Testing
- Recruit more participants: No-shows for remote studies can be higher than in-person studies. In addition, there’s no telling how good the quality of the unmoderated session is until you watch it. Recruiting more participants can help to address these potential problems.
- Filter: Make use of the screening process to filter out any unwanted participants, especially those who do not fit your target audience.
- Practice & Revise: Make sure you always have a pilot test, even if you’ve done it many times before. This will help to iron out any glitches, revise any instruction, tasks or questions that need to be fine-tuned to avoid any potential misunderstandings.
Conclusion
Remote usability testing can be more cost-effective compared to in-person testing, and a larger number of participants can be easily recruited. However, remote usability testing provides fewer data points compared to in-person testing but the insights from remote testing are closer to real-world insights because the participants are usually in their natural environment. Remote usability testing also provides additional flexibility and capacity in recruitment without any geographical limitations, as long as the participants have access to the necessary technology. In saying so, there’s a higher risk of fake participants that want to take part of the study just to make quick money, so there’s a need to take extra precautions in screening and filtering the participants for remote testing.You may find these articles related to Remote Usability Testing helpful!
- Remote Usability Testing 101: What? Why? How? [Infographic Tips]
- 7 Tips to Avoid a Lousy Remote Moderated UX Testing
- The Silver Lining of Covid-19: Empowering Remote User Experience Research
