Usability Testing Guide for Remote and In-Person Test

What is usability testing?

Usability testing is an observational technique to understand and evaluate how intuitive a mobile app or website is based on real user interaction. 

Usability tests are performed to find problems, design opportunities, and learn about user behavior and preferences. It’s an essential UX research technique for happy customers and high conversion rates.

Usability tests involve task scenarios that include a goal and context to help participants resonate with achieving the goal. Below is an example task scenario testing the DoorDash mobile app to observe how intuitive it is when ordering food for the first time.

Source

Users are given a task that involves one (or more) key app processes. They are asked to go through the steps while researchers observe and track their progress. Researchers ask questions or provide tasks that allow participants to express opinions, troubleshoot, and think out loud.

Why is usability testing important?

Validate & adjust features using data

User testing is essential to validate your app design. It provides quantitative and qualitative data, such as KPI metrics and session recordings. Behavior analytics data can also help improve your app design.

CRM and product manager platform JobNimbus experienced first-hand how valuable mobile analytic data is when they used UXCam to help resolve their problem with low app ratings, causing their customers to churn.

UXCam’s product suite helped the product team to understand their customers, giving them the confidence to redevelop the whole app to meet their customer’s expectations based on UXCam’s insights. They successfully took their 2.4-star rating to an impressive 4.8 and retained more users.

Set your product up for success

Usability testing also helps product teams determine how successful their app will be before release. Will users find it valuable and want to use it? Red flags can be identified and addressed before going to market, where they’ll be more expensive to fix, and your brand reputation will start on the wrong foot.

Gain a major competitive edge

It’s better to go the extra mile for a competitive advantage over the other apps in your industry since only 55% of businesses perform UX testing. Usability testing ensures your app delivers intuitive UX. Users will interact more with products and services when they feel understood.

How to conduct effective usability testing

1. Pick a Usability Testing Tool

You need to capture authentic interactions to get the most out of a usability test. Every tap, pause, and area of focus contributes to a comprehensive evaluation of your user’s experience. 

UXCam delivers just that. Thanks to our market-leading SDK, our data is the most accurate for mobile apps. We show you how users experience your app from their perspective through session replays, heatmaps, journey analysis, and many other features.

Consider joining some of the largest mobile app brands in the world who rely on us for actionable findings from their remote usability tests.

2. Define objectives and goals

First, lay the foundation to guide and focus your usability test by clarifying a few points. Ask yourself questions like:

• What are you testing and why? Define the scope and what you hope to get from a usability test.

• What type of users are best to test with? Consider users representative of your customer personas, or you can focus on a particular segment (e.g., active users onboarded within the last 30 days).

• What do you want to observe your users doing? The answer here will later become your task scenario for moderated testing.

Additionally, your groundwork may include collaborating with stakeholders to learn the core elements they want to include as goals and objectives.

3. Plan the test

Planning the details of your usability test sessions well is key to effectively receiving the results you’re looking for and meeting your goals and objectives. Here are the main points to consider when planning your remote or in-person usability test session.

Test environment

Will testing take place at your offices or in a dedicated user-testing lab? For example, a special test environment is necessary for specific conditions and several live observers. 

One of the benefits of arranging a location to test from is that you can control the environment. For example, ensure the room is well-lit and no light is directly above the mobile screen to avoid glare. However, using a location will dip into your budget. Even if sessions are from your office, you may need to pay for their travel and refreshments. Additionally, it’s time-consuming to arrange.

Remote testing is cheaper and more convenient. And there are plenty of tools available to test a more comprehensive selection of people from their natural environment. However, remote testing may not be the best choice for controlled sessions and long, complex tasks.

Timetable

When is the best time to test? 

You’ll need to consider booking the venue and moderators for in-person sessions. For in-person and remote moderated sessions, you must arrange a suitable time for your participants to test.

Moderators

Who should run the test sessions? Is an experienced moderator required, or will arranging basic moderating training for your team suffice? Perhaps moderators aren’t necessary.

Equipment

How will you observe behavior?

Recorded sessions are the best way to thoroughly review after the test. Teams can take their time to scrutinize interactions and capture details that could be missed through live moderation. 

For in-person sessions, you can use video or audio recording, which will require setting up in a way that records the user’s screen and the participant to capture speech and body language. A testing lab can accommodate this.

For remote unmoderated sessions, recording the screen is easy. You can use screen-recording apps or mobile analytics tools like UXCam to record the entire in-app session, allowing you to view the experience from their perspective.

Many apps are available for remote moderated sessions that allow you to interact with and see the participant simultaneously throughout the session. 

Users

Screen users to ensure you have a representative set of test participants for reliable insights. For example, your target customers would be an excellent place to start when testing the usability of a new app. When recruiting users, consider:

• Sample Size: How many people will participate? Most usability testing experts recommend 6-8 people to gain enough insight into user behavior and preferences.

• Segment: Which segment of users are you targeting for testing? It should be the group most closely related to your research question. For example, if you’re curious about how a younger generation interacts with a new app, you’ll want to recruit users in that age range.

• Participants source: Participants can be recruited via social media or a recruitment provider. You may need to incentivize participants to participate (e.g., a gift voucher).

Ensure participants have complete details about the study and consent to be recorded before the session starts.

Source

Consider using a usability test plan template for complete and centralized details.

4. Create a usability testing scenario

Test scenarios set the scene for the tasks you want participants to complete and encourage them to immerse themselves in them. They should mimic real-life scenarios of how the user would use your app to achieve something or solve a problem. 

Ensure the scenarios and goals you want your participants to complete are well thought out and align with your objectives and goals. Click here for free usability testing templates to help get you started.

5. Conduct the tests

The session should be mapped out for in-person testing so it flows and moderators and participants know what to expect. Moderators should be clear on how to facilitate the session, the order of tasks, when to stop for breaks, etc., to guarantee the session stays on schedule. 

Session plans for a moderated session may include the following:

• Introductions and Warm-up: For in-person testing, ensure the participants are comfortable. For remote sessions, check whether you can hear and see each other and that all the software works as expected. Remind participants the session will be recorded and ask them to sign the consent form.

• Pre-Test Data Collection: Gather more demographic and psychographic information for valuable context for analysis if necessary.

• Start Testing: At this point, the rapport you’ve built with participants[s] should make it easier to transition into the first task. 

• Follow-up Session: Allow time at the end for follow-up questions and to gather user feedback. 

Analyze and present test data

Once all the testing is done, it’s time to analyze the results to find actionable insights and bridge the gap between real-world knowledge and the solution. The answers you’re looking for will be in the data.

Results can be categorized into quantitative and qualitative data, making it easier to feed your findings company-wide. For example, you could present the results for the metrics analyzed during the study, so it’s easy to see what happened and areas of concern (e.g., task completion and time on task, as shown in the table below).

Source

Qualitative data is taken from session recording observations and direct feedback.

UXCam does an excellent job of creating AI-generated reports for your remote usability tests. Our dashboard information can answer your questions in seconds and show you the “why” behind your quantitative results. Dashboards can be shared internally and externally for seamless collaboration and shareholder opt-in.

Optimize for usability with UXCam 

Usability testing highlights the parts of your app design that users find easy to use and the challenging areas. There are a few options for the best way to conduct them. Ultimately, your decision depends on time, resources, and the best way to achieve your goals and objectives for the test.

With UXCam, you’ll access tons of quantitative and qualitative data based on all your user’s innate interactions and gestures. Our session replay, heatmaps, and segmentation  functionality show you your customer’s experience from their standpoint. We highlight behavior trends and patterns for a 360-degree understanding of your app’s usability.

Get started with a free trial and see what UXCam can do. You might also be interested in these;

FREE usability testing templates and how to use them

6 Usability testing methods to improve UX in mobile apps

How to conduct mobile app usability testing: 2023 Guide

5 Usability testing examples in mobile apps

5 Fantastic remote usability testing tools you can use now