If you are new to usability testing, you’re highly encouraged to read Usability Testing: What Why How beforehand.
In this article, we will cover Remote Usability Testing 101 on:
- What is Remote Usability Testing?
- Why Remote Usability Testing?
- Is Remote Usability Testing Effective?
- How to conduct Remote Usability Testing?
- Tips for remote usability testing (infographic)
What is Remote Usability Testing?
The main difference between remote testing and a traditional lab testing would be the location of the tester. Remote usability testing requires a user to proceed to a lab or an isolated location (such as home) to do testing.
At its core, testing is conducted by getting a tester to install a recording software or plugin on their laptop or computer to record both their screen activity and also their voice when they are conducting a test. Upon finishing the test by completing a series of task, a video recording will be generated. This video is then analysed and shared with team members.
There are other types of remote testing services in the market too, but this article’s main focus will be on unmoderated remote usability testing.
It can be difficult to source for the right candidates to conduct lab testing. Other than budget concerns, time has to be sacrificed in recruiting participants and scheduling suitable times to conduct testing. The process can be very tedious. In addition to that, the participants might not feel comfortable or convenient to do the test in a specified location.
Previously, when we conducted usability testing through lab or guerrilla testing methods, we found that there were numerous flaws. Some users tend to misbehave during the test while others felt uncomfortable or behaved out of the ordinary due to the awareness of being observed by someone.
This is understandable, as we don’t usually have someone sitting next to us telling and observing how we use a particular website or product – these reasons are why remote usability testing exists.
Remote usability testing enables users to conduct tests at their most comfortable atmosphere & setting (preferably, with their own equipment: laptops, mouse, mobile phones etc). For instance, one would certainly prefer sitting at their desk at home, using his or her own laptop to browse a hotel-booking site.
This method is the preferred ‘natural’ way of obtaining user insight. With a remote usability testing tool, users can literally conduct tests anywhere, anytime at their wish. This is also the main reason why many corporations opt to conduct remote usability testing.
From our experience, inviting users to a lab to conduct a test using unfamiliar devices that do not belong to them tends to complicate matters. This is a problem that can and should be avoided when planning for a test session.
The picture below illustrates a scene where a user was asked by a UX researcher to get comfortable in the test lab, the user ended up getting a little too ‘comfortable’. You can read more about some of rib-tickling scenes that happened in lab testings or focus group sessions here.
Is Remote Usability Testing Effective?
A study conducted by STRATO shows that usability testing as a compliment to traditional laboratory tests can be used, for both methods were able to identify the same problems. STRATO AG is an internet service provider headquartered in Berlin, Germany. It is a subsidiary of Deutsche Telekom AG.
Identical Results at cheaper costs and in half the time
STRATO’s Usability Department, under the direction of Manuel Panzirsch compared results of their in-house (laboratory) usability test with the results of Rapid User Test in an effort to improve the quality of Rapid User Test’s service.
Two sets of ten test subjects between the ages of 18 and 40 were utilised for the experiment. One set was instructed to do a remote usability test of STRATOS’s online storage service, ‘HiDrive’ whilst the other set of ten were asked to conduct the test in a laboratory.
After completing a list of three tasks, the results were clear: both remote user testing and in-house testing identified the same usability problems.
In both tests, 10 usability problems were identified, of which 9 were identical. In addition to that, remote usability testing was able to identify an extra usability problem, whereas the other method failed to do so.
Remote Usability Testing vs. Classical Laboratory Testing
Rapid User Test (our partner in Europe) produces same results as in-house usability testing
In half the time and two-thirds of the cost of laboratory testing, Rapid User Test has proven that remote testing produces results that do not fall short from the standards of traditional laboratory tests.
“We have conducted a comparative study with Rapid User Test. Remote usability testing produced very good results and was able to uncover similar usability problems with traditional in-house tests. The services offered by Userlution will be invaluable to us and in addition to traditional in-house usability testing, will create an important tool in making user testing affordable, fast and reliable.“ – Manuel Panzirsch, Product Management of Head of User Experience
The differences between the two methods were able to be identified in the comparative study. The ten remote testers supplied by Rapid User Test took a much shorter time to review the website (15 minutes versus 45 minutes used by in-house testers), which suggests that the reduction in involvement of an invigilator saved time.
However, it is questionable as to what extent the significance of an invigilator is – whether one is required to ensure a test is conducted based on a thorough methodological system, or is disadvantageous due to an invigilator influencing the behaviour of a user testing a website.
Obtaining meaningful results
It is of paramount importance to formulate a clear list of tasks for remote testing so that tests can be conducted without compromising the reliability of its results. Remote testing is a method that is able to identify critical usability problems at a low and affordable cost.
How to conduct Remote Usability Testing?
Since no one is moderating the test session to guide the users, drafting a good test plan is essential. Before you release the test to the targeted users, it is advisable to conduct a pilot test to check whether the test plan that consists of different tasks is easily understood and not misleading.
For instance, when a user was asked to open a new tab and ‘continue’ browsing the website, the user might get confused on whether he should continue browsing the initial website or the newly opened website on the new tab. The test instruction has to be very clear & straightforward, so that it minimizes doubt for users.
The think-aloud protocol plays an even more substantial role in remote usability testing. Again, there will be no moderator there to probe the user for deeper insight, so you must ensure that the tester is able think aloud while doing the test and your task questions have a ‘probing’ capability.
Users have lots of experience thinking in their mind, but they are less likely to express everything verbally and openly unless they’re directed to do so. This is especially common attribute of Asian users who tend to be more ‘polite’ and critique less.
Certain users will naturally stop or abandon a task that is required of them to perform if your tasks are not well presented in words. You need to have a certain level of good UX for your task description as well.
For example, if your main objective is to understand the user interaction on the whole checkout process of an e-commerce webpage, you should describe something like “Please go through the whole checkout process, you can stop only when you’re asked to provide your credit card details”, after directing the users to choose an item of their choice. There are times where users could not even manage to reach the intended page to input credit card details.
Although the ‘think-aloud’ protocol is an important criteria in usability testing, it is not everything. Users might not know they are doing something wrong despite them thinking and expressing themselves verbally out loud throughout the whole process.
One example of this is when conducting a test, a user commented that he was able to complete the task easily and remarked that the website was easy to use during his first impression as well as post-testing questionnaire. However, the client was surprised to see that the user had used the site’s function in completely wrong way after watching the test video.
After receiving insights on usability issues faced by users through testing, you can also complement the result with other quantitative analytical or testing results. Instead of knowing only just how (something happened), now you can understand better why users do not convert at your website or why there is a high bouncing rate at certain pages.
Bonus: Some quick tips for you!
Below we will share some tips for you to conduct remote usability testing effectively, from setting your test objective to drafting your test plan (unmoderated).
Setting Your Test Objective:
- Understand Your Challenges
- Cross-Functional Communication
- Recruit Right (Number of) Testers
Draft Your Test Plan:
- Be Task-Oriented, Ask Non-Leading Questions
- Probe Users to Think Aloud
- Optimize Your Task Instructions
- Don’t Focus Only on Known Issues
Last, But Not Least:
- Ask Follow-Up Questions
- Run A Pilot Test & Amend Test Instruction
- Share The Video Highlights
Interested to conduct a remote usability testing for your site, prototype or even your competitors’ site? Why not set up a test on NetizenTesting.com today with our ready-made templates. Set up a test, sit back, take a break, receive your video results, share it with your team and fix it like a secret agent with mind-reading power.
