We rely on their mobile devices to stay connected and get work done. But how do you know if your mobile app or website is truly user-friendly? The answer lies in mobile usability testing.
Mobile usability testing is a powerful tool that helps companies identify potential issues with their design, functionality, and user experience. In this article, we’ll explore the benefits of mobile usability testing, different types of tests available, and how to set up an effective test environment.
What is Mobile Usability Testing?
Mobile usability testing is the process of evaluating how users interact with a website or app on their mobile device. It helps companies identify any design, functionality, or user experience issues that may be preventing their customers from having a good experience.
During a usability test, an evaluator studies how people use the site or app and makes note of what works well, what doesn’t work well, and where there is room for improvement. By uncovering any potential problems early on in the development process, you can make changes before you launch the product to ensure it meets customer needs.
In some cases, test evaluators use cameras to capture the movements of test participants. This kind of body language can help the testing team understand how people intend to use their mobile devices.
Why is Mobile Usability Testing Important?
Mobile usability testing is more important than ever because mobile usage is increasing. The number of smartphone users has increased by 87% in just seven years. So it’s essential to ensure that these mobile experiences are user-friendly, efficient, and effective.
A positive user experience is critical to the success of any mobile application or website. Usability testing can help identify areas of the interface that may be confusing or frustrating to users, allowing for improvements to be made.
Additionally, mobile usability testing is cost-effective. It’s less expensive to identify problems and fix issues in the design phase of a mobile application or website rather than after it’s launched. Usability testing can help catch issues early in the development process, ultimately saving time and money.
Furthermore, mobile usability testing can give you a competitive advantage. It can help you differentiate your brand from your competitors’ brands by offering a more user-friendly and enjoyable mobile experience.
There’s also the added benefit of learning about your customer. You can obtain valuable data on what your customers like and don’t like, even if your tests don’t inspire significant changes to your site or app. Knowledge about your customer can be used in other marketing initiatives.
Most importantly, the effectiveness and ease-of-use of your mobile site or app can affect your brand reputation. A poor user experience damages the brand, but a positive user experience can improve brand loyalty and customer satisfaction.
Types of Mobile Usability Testing
In all mobile usability testing, users are usually given tasks to complete on the app or website being tested and their actions are monitored and evaluated by researchers. Usability tests are moderated or unmoderated. They can be in-person or remote.
Moderated testing
Moderated usability testing refers to a test conducted under the oversight of a moderator. The moderator gives the tester tasks to complete. The moderator can ask questions or probe for more information based on the tester’s behavior. Actual user feedback can be quite helpful to your future design process.
During moderated testing, researchers can get a better understanding of how users interact with the product. The downside is that it’s expensive and time-consuming to have a moderator sit with every test participant.
Unmoderated usability testing
Remote unmoderated testing is a type of mobile usability testing that allows companies to test their products with real users in their natural environment. The user completes tasks on their own devices and researchers are able to monitor the user’s interactions and evaluate how they interact with the product.
This type of testing can be incredibly beneficial for companies as it provides valuable insights into how people use their products outside of a controlled lab environment. It’s also inexpensive and easy to conduct at scale. However, the participants don’t have the opportunity to ask questions and there’s no moderator to probe deeper into their behavior.
In-person vs. remote testing
In addition to deciding whether to conduct moderated or unmoderated user testing, we will also need to decide whether you will administer the tests in-person or remotely.
In-person usability tests (often called lab usability testing) are helpful because you can control the environment in which the user accesses your mobile site or app in a lab setting. You could supply them with a device so all the tests are the same.
In-person tests also let the participant form a connection with the moderator (if you’re using one), which might encourage them to reveal more information.
Remote testing is helpful because it gives you access to a wide pool of test participants. It’s also inexpensive, even if you’re using a moderator. You’ll need to find a mobile usability testing tool that gives you access to the participants’ device screen (or a screen recording).
In-the-wild user studies (guerilla usability testing)
Guerilla usability testing is another option that’s right for some organizations, though admittedly it’s a niche case.
These kinds of tests are conducted quickly and informally, often in public places, such as coffee shops or parks, with little or no advance planning. This type of testing is usually done using a small group of participants, and the focus is on identifying the most significant usability issues in the product.
The process typically involves asking random participants to perform a set of tasks while being observed by a moderator, who may ask questions or provide feedback. The main benefit is that you get feedback from people who don’t have any experience with your site or app. The downside is that your participants probably won’t align with your target audience.
Target Audience for Mobile Usability Tests
Choosing the right audience for your mobile usability testing is a crucial part of the process. You can’t just pick any random person to take part in your studies. Instead carefully consider who their target audience is and how they should participate in your tests.
For example, if your company sells fitness equipment, you should focus on not just users that are interested in fitness, but people who have purchased similar products in the past. If you sell cardio equipment, you shouldn’t target people who only lift weights. This is the only way to ensure that your mobile usability tests will yield accurate data that is relevant.
Furthermore, this helps ensure that the results of these tests are reliable and useful when making changes to your mobile site or app. By taking the time to assess your target audience and user groups for mobile usability tests, you can ensure that the data gathered from these tests is accurate and useful.
The Challenges of Mobile Usability Testing
Mobile usability testing is a powerful tool to evaluate the effectiveness of your mobile site or app, but it’s not without challenges. Here are some obstacles you should be aware of before you implement a mobile usability test.
Choosing the right target users
Selecting the right test participants is difficult and tedious, but it’s critical to the success of your test. If you test your mobile site or app against the wrong audience, your data won’t be as valuable. You want your site to perform well for real users.
Device/software fragmentation
There are a wide variety of mobile devices with different screen sizes, resolutions, operating systems, and versions. It can be difficult to test on all of them, and testers must decide which devices to prioritize.
Security and privacy
Mobile devices contain sensitive information such as contacts, messages, and personal data. Testers must ensure that the testing process does not compromise the security and privacy of users’ data.
Cost/expense
If you plan to conduct moderated mobile usability testing, you may find it expensive to pay the evaluator to sit through every test. If you want a lot of data, the cost can rise quickly. Furthermore, since the data is qualitative, someone has to spend time going through it and extracting insights, which is time consuming and expensive.
10 Steps to Conducting Mobile Usability Tests
Now that you understand how they work, let’s talk about designing your own mobile usability tests. The results of your test can only be as valuable as the test itself.
Step 1: Set a test objective
Your first step is to set an objective for your mobile usability test. What do you hope to achieve? In some cases, you may be looking for hidden errors or points of friction that you wouldn’t uncover on your own. In other cases, you may have specific concerns, such as the functionality of a site feature or the usability of a new page.
Here are some common questions that usability tests can answer:
- How fast can users learn how to use a page?
- How many mistakes do users make trying to achieve a goal? How serious are those mistakes and how easily can they recover?
- Once the user knows how to perform a task in the app, how fast can they repeat it?
- How quickly do they regain proficiency when they return to the site or app?
- What do they like most and least about the site or app?
Understanding the overall objective will help you formulate your test tasks, decide whether to use a moderator, and set a budget for the testing process.
Step 2: Define your segment and recruit
Depending on what you hope to achieve, you may only need to run a test for a segment of your actual users. For instance, suppose you want to run a mobile usability test on your furniture store’s new standing desk customizer feature. It would make sense to recruit participants who use a standing desk every day, rather than people who prefer to sit at their desk or don’t work at a desk at all.
The Nielson Norman group suggests that you only need five users for any type of qualitative usability testing. That may seem like much, but it’s all you need to identify usability issues. Interestingly, testing more users doesn’t result in appreciably more insights.
Step 3: Plan your tasks
Since mobile usability testing involves asking participants to complete a series of tasks, you have to generate a list of tasks. Tasks are generally a single phrase with brief instructions. Here are some basic task instructions:
- “Log into your account.”
- “Navigate to the bedroom furniture page.”
- “Add two of the same product to your shopping cart.”
- “Find our shipping terms.”
Your task instructions should be clear, but simple. You don’t want to give participants too much explanation, otherwise you’ll walk them past any potential problems.
Aim for tasks that test the site or app’s main functionalities, such as navigation, filters, product pages, adding and removing items from the cart, upsell widgets, and checkout. You could also ask test participants to find key pieces of information that shoppers typically want to know.
Step 4: Create test documentation
It’s important to maintain detailed documentation on every test you conduct. This ensures that everyone on your team understands the nature of the test. It also makes sure the moderator (or multiple moderators) conduct the same test every time.
If this is your first time running usability tests, it’s okay if your documentation is light. Contribute to it over time as you conduct routine tests.
Step 5: Choose a testing environment
Where will you conduct the test? If you’re hosting an unmoderated test, you could simply send the instructions to your participants and ask them to make a video recording. If you’re hosting a moderated test, you’ll need to arrange meetings with the moderator. If you intend to conduct tests in-person, you’ll need to organize a testing event.
Here’s a sample of some basic usability test documentation:
Step 6: Collect device data
The mobile device market experiences extreme device fragmentation. There are several popular operating systems and countless device models. Your test participants will come with a wide variety of devices, so you’ll need to understand what they’re using before the test starts (and that they’re using real devices). This can help you identify a problem that only manifests on a specific device, browser app, or operation system version.
Step 7: Conduct your tests
Once everything is in place, your next step is to administer your testing sessions. Choose whether you’ll conduct tests with or without a moderator and either in-person or remote.
Encourage participants to think aloud as they complete the tasks, even if there’s no moderator present. Simple phrases like, “I didn’t expect it to do that” or “Now where is that link…” can be extremely helpful to understanding their cognitive process.
If you’re a moderator, encourage him/her to be patient and non-judgmental. Most importantly, this person should not lead the test participant to the answer.
Here are a few more tips from the Interaction Design Foundation.
Step 8: Analyze the test results
Once you’ve finished conducting your tests, it’s time to assess the data. Review your notes, video recordings, written feedback, and the completed or unfinished tasks.
Ideally, you should go over test sessions one-by-one. Examine them each alongside some other people on your team. It’s helpful to have multiple heads working to understand the user.
Identifying problems is critical, but you should also identify what works and try to explain why. This information could help you optimize other parts of your site or app. For instance, if customers are drawn to a particular image, you could use similar imagery in other places.
Collect everything you learn in a central place so it can be accessed by everyone at later points. If you haven’t started a repository of information on your customer, this should be it. It’s a good habit to turn your data into a standardized report. This will let you easily compare changes over time.
Step 9: Make improvements on your site or app
Now that you’re armed with valuable information, it’s time to start making changes to your site or app. Start with the basic stuff that should obviously be changed, like 404 pages, broken links, spelling errors, or out-of-place elements.
Then make a list of potential changes based on user feedback. For instance, if users complained that they couldn’t find the “add to cart” button, consider moving it to a more obvious location. If they reported being unable to find your shipping information, prepare an experiment to test it in different locations.
Next, prioritize your changes based on their impact on the user experience. If a task will help a majority of your users or have a big effect on conversions, place it at the top of the list.
Finally, delegate the changes to the appropriate people within your organization. You may have to send action items to developers, designers, or data analysts.
Step 10: Plan your next mobile usability test sessions
Mobile usability testing is something you should be doing regularly. Mobile usage is only going to increase in the future. New web users and ecommerce shoppers will enter the market through their mobile devices, so you should expect to run routine tests to ensure that your mobile site and/or app are working properly.
Furthermore, you should expect to run tests whenever you launch a new feature, build a landing page, or implement new marketing campaigns. A few quick tests to ensure that everything works and your site/app doesn’t introduce friction can go a long way toward building a solid user experience.
Boost Conversions with Usability Testing
If your site has usability problems, potential buyers will struggle to find the products they need and make their way through checkout. A regular mobile usability testing program will help you fix problems and build a mobile site or mobile application that creates a positive experience for your users and better revenue for you.
Does mobile usability testing sound hard? You aren’t alone. Designing tests, recruiting reviewers, and facilitating the sessions is time consuming and challenging if you haven’t done it before. Let UserInput provide fast, reliable user testing that won’t blow up your budget.
Our managed user testing service identifies technical bugs or points of friction, so every campaign you run has the best chance at success. Start your project today.
