Remote Moderated Usability Testing is a great way to get quick valuable insights without having to set up a full test lab or rent an expensive external testing facility. Even though it’s a convenient way of conducting research, careful preparation must be done to reduce disruptions, maximize time, and keep the study consistent.
Here are 7 simple tips you can follow for remote moderated usability testing:
1. Prepare for Disasters
Similar to preparing for in-person usability testing, backup plans are vital to have in case anything goes wrong. Typical things that may go wrong include: poor internet connection, poor audio quality, limited video connectivity, delayed start times, and no-shows.
To reduce the chances of poor internet connection, consider screening participants to mitigate these issues. You may include a question in the screen regarding their monthly internet speed to determine if their internet speed is sufficient for the remote moderated usability testing.
During the session, if the audio quality is playing up and you’re having a hard time listening to your participants, a good backup plan to have in this situation would be to ask your participant to use their cell phone for the session. Make sure you have a detailed step-by-step instruction guide for participants, like the dial-in number to call, or adjusting the audio settings.
Unfortunately, late-comers and no-shows aren’t something you can control but there are things you can do to prepare for this. Recruit extra participants just in case, if there’s a no-show, you won’t need to scramble to find a replacement. If you need 5 participants, recruit an extra participant, the extra person gives you a comfortable buffer for any no-shows.
2. Show Up Early
Aim to show up a bit early, this will help alleviate any wasted time from tech issues. Ensure you have sufficient buffer time at the start of the session for tech checks, instructions or introductory materials, 15 minutes would be sufficient.
For remote moderated usability testing, make sure that both you and the participant show up at 10 minutes before the session start time to set-up, to do a quick tech check, and share any instructions before the session begins.
3. Interrupt if Need Be
It isn’t considered a best practice to interrupt your participant, especially when they are speaking, thinking out loud or completing tasks. However, when there’s a technical issue or if the participant doesn’t quite understand the instructions given, you may intervene.
If there’s a technical issue, firstly identify the source of the problem and check that it isn’t on your side. If the issue is indeed from the participant’s side, politely ask them to repeat themselves and explain clearly that the audio was cut off.
If the problem persists, ask them to check the internet speed to ensure their connection is solid. Again, do this only after checking that the problem doesn’t stem from your side.
In the event that the participant doesn’t understand the instructions given, politely interrupt them and clearly state that you made a mistake in explaining the instructions. Yes, even if you didn’t make a mistake, do this. Avoid making the participant feel as though they had misunderstood or they will rely on you for more guidance throughout the session.
It is OK to interrupt participants if they have said something you would like them to elaborate on. For in-person usability testing, it tends to be easier to follow the participant’s thought process but for remote moderated usability testing, nonverbal cues and body language isn’t readable. So if you need to be certain about what was said or what you observed, it is okay to interrupt your participant.
4. Remember to “Unplug”
You know when you’re on a video call and suddenly there’s a random notification sound? It’s a slight distraction but for remote moderated usability testing, we want the participant to be able to have full focus and attention.
Before the session starts, make sure all your notifications are turned off so that there aren’t any unwanted chimes or alerts which will be distracting and poses a risk of interrupting or even influencing the participant.
Remote moderated usability testing can be so quick and easy to set up, but don’t forget to block out your calendar so that relevant stakeholders and colleagues know that you’re conducting a usability test session and should be undisturbed.
It also helps to put a sign outside the room to inform people not to interrupt and to keep the noise level down as there’s a session in progress.
5. Make Use of the Chat Function
Often times, words spoken are placed into the short term memory and participants may forget what you said earlier on. In using the chat to post tasks, the participants will rely more on the written task instead of relying on you for information during the session.
In addition, any scenarios, links or step-by-step instructions can also be posted on the chat, this is an easy reference point for both parties and it’s so convenient to find. Participants are then able to revisit this information whenever they need to, and work at their own pace instead of trying to recall everything from memory.
If the participant is required to use dummy login credentials or a faux credit card, this is especially important because this information is heavy on the memory. Instead of being verbally communicated and risk miscommunication, the information is clear and easily accessible.
In saying so, don’t overcrowd the chat function with all the tasks and information at once. Try to work with your participant’s pace and provide the information accordingly. It helps to have an existing document that you can copy and paste from, and be sure to use simple language and instructions to avoid confusion.
6. Take Notes
During an in-person usability testing session, taking notes in front of the participant isn’t considered best practice, it can create an invisible barrier between you and the participant.
However, remote moderated usability testing doesn’t impose such a barrier because you’re already conversing remotely. If you find that you can’t focus on the session and take full notes at the same time, don’t.
You can try to document key interactions and highlights but don’t risk the attention you should be paying to the participant. Consider inviting a note-taker to observe the session to take detailed notes instead of dividing your attention.
7. Avoid General Population Recruitment
Don’t fall into the trap of recruiting only the general population, it may be an easier alternative and so convenient when you’re conducting remote moderated usability testing but they aren’t your target users.
Make sure you recruit participants that are representative of your target users, these are people who have true motive and mindset in completing the tasks on the product being tested. People pay attention to different things depending on their goal, and if their goal is without motive, they will pay attention to very different things compared to your real users.
Representative participants will pay closer attention to the content, meanings and association when they are navigating and completing tasks.
