How to Conduct Usability Testing for Mobile Apps?

How to Conduct Usability Testing for Mobile Apps?

Feb 15, 2022 | User Testing, UX Guide | 0 comments

During usability testing, the moderator typically collect qualitative data like behavioural observations and participant comments. UX Researchers would also sometimes collect quantitative data like task times and success rates. 

By critically analysing this data, the UX researcher seek to identify usability problems within a mobile application or website. 

In today’s blog post, we shall delve into usability testing for mobile applications and its underworkings.

What is mobile usability testing? 

Mobile usability testing is essentially usability testing performed on mobile products and applications running on mobile platforms. 

This usability testing exercise attempts to determine how end users might perceive a mobile app or product, either in a usability laboratory or in a field setting.

Is mobile app usability testing important? 

Mobile app usability tests seek to observe test-subject users while utilising a specific app. The core purpose is to measure the application’s user-friendliness to better support a brand’s key commercial objectives.

Mobile usability testing also ensures an app adds value to a business, thus meeting the expectations of the final users. 

Generally, ensuring good usability for mobile apps helps improve customer satisfaction, decrease time spent on customer support, and increase overall sales and revenue.

Furthermore, a well-tested and improved application attracts visitors’ attention and helps set up trust and cooperation between the app owner and the app’s potential users.

usability testing mobile app

How to run effective usability testing for mobile apps

Step 1: Define objectives

Before commencing any usability testing exercise, you should set your objectives and goals straight. In essence, you should clearly define the questions you want to answer with the usability test. Or even clarify the hypothesis you want to test with the usability test.

As you define the test’s objectives, ensure you explore these important areas:

  • The app’s roadmap (for example if something is important to be tested or it will already be removed in the next update)
  • The potential impact of the test
  • Existing users and markets for whom the app is targeted
  • The app’s existing competitors
  • Timing and scope

Step 2: Design the tasks

Once the test objectives have been defined, you then need to set the tasks of the usability test. Tasks should be one sentence long and consist of the interactions to be performed by the test users, for example:

  • Register an account
  • Upload a photo
  • Accept a friend request
  • Sign into an account

However, rather than directly asking the test user to execute a task, tasks should be converted into task scenarios. These provide more context to participants about why they are doing the task. And thus, facilitate more natural interactions similar to what an ordinary user will perform with your app.

Consequently, task scenarios that are defined should always be:

  • Realistic, actionable and without any obvious clues about how to perform the steps.
  • Sequentially ordered to ensure a smooth flow of the test session.
  • Tied to one or more objectives.

 Step 3:  Prepare the usability test documents

When conducting usability testing for mobile apps, there are several documents you typically require. For example: 

  • Consent forms (for minors and even adults)
  • Post-test questionnaire 

Ensure these documents are well prepared to suit your user group and their nuances.

respondent signing usability test consent document

Step 4: Prepare the test participants

Mobile usability testing typically involves real users undertaking realistic tasks that the mobile app is intended to achieve. 

As you prepare to test with real users, there are several considerations one can take when vetting participants: 

  • Choose users who are a fair representation of the target audience.  
  • Users own a mobile device with the exact operating system (including the appropriate version/s) that is being targeted.
  • Participants must be available at the time, place, frequency of the intended usability tests.
  • Users must agree to the compensation terms that you’re offering (if any)
  • Participants must be ready to sign a usability test participation consent form

Step 5: Choose a mobile application usability testing methodology

Principally, there are two main methods for conducting usability testing of mobile applications. These are:

  1. Laboratory-based usability testing
  2. Remote usability testing

In this step, weigh the pros and cons of each approach and then pick one that will fit your needs. 

Step 6: Reporting the results of the usability test

After collecting your usability test data, the next step is to compile, organise and analyse it to draw meaningful conclusions. 

The data can be split between quantitative and quantitative data. For instance, quantitative data encompass completion rates, task times, success rates, satisfaction ratings and error rates. 

Or qualitative data like problems experienced, answers provided in the questionnaire, post-test interviews and debriefing sessions.

This step basically involves careful analysing notes on recordings, transcripts, and other information you might have gathered. It then moves on to presenting it in a way that delivers actionable recommendations.

Conclusion

Usability testing is vital to the success of mobile apps. Mobile apps that users perceive as easy to learn, user-friendly and less time-consuming tend to be more profitable and popular. 

Furthermore, usability testing should occur on a regular and scheduled basis, especially when introducing new design features, or updates to improve functionality.

The article is a part of our comprehensive guide on “Usability testing”.

Field Studies vs Usability Testing: Which Is Better?

Field Studies vs Usability Testing: Which Is Better?

Feb 14, 2022 | User Research, User Acceptance Testing, User Testing | 0 comments

Usability is essentially the measurement of a product’s usefulness from the users’ perspective. In principle, usability can be segmented into three fundamental elements: efficiency, effectiveness, and user satisfaction. 

As a research methodology, the lack of usability testing during the development of a product often leads to user dissatisfaction and rejection. Another research methodology that serves a similar purpose of ensuring product utility is a field study. 

What is the difference between field studies and usability testing?

Field study encompasses all studies of users in their natural environments (usually conducted in the user’s context and location as opposed to your office/lab). 

For example, homes, workplaces, neighbourhoods, parks, streets, and shops. Essentially, localities and areas in which one’s product might eventually be utilised are the best places to conduct these studies. 

On the other hand, usability testing evaluates a product by directly testing it with representative users. 

During a usability test, participants attempt to complete typical tasks while testers/observers watch, listen and take notes. 

Overall, the core goal of usability testing is to identify usability problems, collect qualitative and quantitative data to determine the participant’s satisfaction with the product.

conducting remote online usability testing on laptop

When to choose a field study? 

Field studies deliver a complete, unbiased picture of what prospective users actually do with a product. They provide context, especially if a product is designed to function in a particular context. 

In such instances, conventional lab testing might not give you accurate results. So, when your usability research cannot fit in a lab, you may need to carry these out. 

Furthermore, field study is well suited for practical products. For instance, if a device is engineered to function as a core component of the navigational system of an oil tanker. Then visiting the oil tanker for final testing makes sense. 

Some of the examples of field studies include user’s office/home visit, ethnographic research, and contextual inquiry. 

How to conduct a field study? 

1. Preliminary planning 

This period involves working with participants. You don’t necessarily require a specific set of research questions but defined study topics. Field studies are strictly observational, as researchers aim to be as unobtrusive as possible. 

2. Direct observation

This part of a field study involves watching users critically observe how they behave and why. Ideally, the test subjects do not care that you’re watching, and act exactly as if you are not there. 

3. Participant observation

Here, the researcher can join the group of people being studied and records data as field notes or diary entries, after finishing observations for the day.

Benefits of field studies in UX design

Field studies enable you to understand your users in-depth, under realistic conditions. Field studies also allow you to discover social defects and understand environmental factors before releasing products.

Generally, field study advantages can be summarised as:

  1. It yields highly detailed data.
  2. It emphasises the role and relevance of social context.
  3. It can help uncover social facts that may not be immediately obvious or that research participants may be unaware of.

In person field work study usability testing on mobile app

Potential pitfalls of field studies in UX design 

The key disadvantage of field studies is their expense. Furthermore, most field studies can’t usually be automated or sped up with technology. This is because they typically rely on old-fashioned theories of patience and observation. 

Additionally, if your study design requires a large number of observers, then a field study might not be ideal. Similarly, if your product is to be used in rare, unpredictable circumstances (for example, first-responder mobilisation after an earthquake), then you can’t do a field study.

Further, if your study involves collecting sensitive, confidential information, you might require the more controllable circumstances of a user research lab.

When to choose usability testing?

Usability testing is ideal when: 

  • Seeking to uncover any issues within your design, workflow, or process.
  • Validating if a design works for participants – especially if there is debate on a project team about how something is ‘utilised by users’.
  • Seeking insights into both good and poor issues with an interface.
  • Seeking different perspectives and mental models on an interface.

You can choose to conduct usability testing:

  • Before significant design decisions are made.
  • In high-risk, low-certainty situations.
  • When it’s time to evaluate and iterate.
  • After product launch.

Benefits of usability testing in UX Design

  • Improved user experience and the product utility.
  • Discovery of hidden usability issues.
  • It ensures that the application’s functionality matches the requirements.
  • It identifies changes required to improve user performance and satisfaction.
  • Helps to analyse product performance to determine if it meets the defined usability objectives.

Disadvantages of usability testing in UX design

  • There is sometimes uncertainty about what to test
  • There can be testing fatigue
  • There is sometimes too much feedback gathered

Field Studies vs Usability Testing Cost

Field studies are expensive and can be highly time-consuming. This is so because of the need to travel, the number of hours researchers are required to commit, and the complex analysis that open-ended, unstructured research dictates.

On the other hand, usability tests are relatively inexpensive and easy to conduct, especially if conducted remotely. However, some of these can only be conducted in a specially designed laboratory, enabling facilitators to interact with, and observe users. Such tests can involve some cost, but they will still be cheaper than field testing.

User Testing Services in Asia

NX logo
NX logo

User Testing Services in Asia

Key Highlights
  • Provides user testing services for websites, mobile apps & other digital products
  • Recruit users who match the client’s target audience & provide them with tasks to complete
  • Users are asked to think aloud and provide feedback on their experience
  • Helps conduct user testing sessions remotely, using a variety of tools & software
  • Provides a dedicated project manager to oversee the entire process
  • Project manager coordinates the recruitment of users, schedule of testing sessions & the delivery of the results
LEARN MORE
CONTACT US

Conclusion

In summary, usability testing is an inexpensive means of gathering valuable feedback from representative users. 

In contrast, field studies allow user researchers to gain first-hand experience and knowledge about the users and the processes they study.

The article is a part of our comprehensive guide on “Usability testing”.

10 Usability Testing Tools to Try in 2022

10 Usability Testing Tools to Try in 2022

Jan 31, 2022 | User Acceptance Testing, User Testing | 0 comments

Choosing an appropriate usability testing tool is vital to executing an effective user experience. Typically, the choice of tool depends on the type of usability testing method you’ll employ.

It is true that you can run usability testing without a specialised tool, for example, when doing qualitative, in-person usability testing. However, if you are planning remote usability testing sessions, or seeking to collect quantitative data, then a dedicated tool is necessary.

What are usability testing tools?

Usability testing tools are dedicated solutions that essentially streamline usability testing activities. These solutions can be utilised to gather actionable insights into how prospective users might perceive and experience your product. 

For example, they can be used to put a prototype or website in front of real users. Thus, enabling website owners to analyse how users accomplish a given task or to identify possible frustration points. 

Read our article on “How to conduct usability testing for websites?”

Generally, usability testing tools smoothen the process of accessing and tracking users’ feedback. They also help with the analysis of that feedback to enable product owners to make data-informed decisions and deliver the best user experience.

Top 10 Usability Testing Tools 

1. Maze

Maze is a rapid user testing platform for collecting both qualitative and quantitative usability data. This platform seamlessly integrates directly with Adobe XD, Marvel, Sketch Figma, InVision. 

Furthermore, Maze allows users to create and run in-depth usability tests, then share those tests with testers via a link. Maze includes varied features like task analysis, guerrilla testing, multiple path analysis, heatmaps, A/B testing, and wireframe testing. 

Maze also generates an instant usability test report for each test that users can share with anyone via a link. 

This reporting functionality presents usability testing results like completion rates, misclick rates, and time spent. Additionally, Maze allows users to run research surveys and test their information architecture using Card Sorts and Tree Tests.

2. Lookback

This is a user experience (UX) screen recording tool for UX designers and product managers who seek to analyse how users interact with their applications. 

Lookback’s testing capabilities enable designers to view what users see and get their reactions in real-time. This is either recorded or in-person. 

For instance, if you set up a remote test, participants shall receive a link to download your app. Then they get started with a live session or self-test, and you can even communicate directly with them. Subsequently, user recordings automatically appear in a dashboard where one can organise them into groups and create highlights. 

Lookback

3. Userlytics

As one of the most popular usability testing tools in the market, Userlytics a UX research service that offers picture-in-picture user recording (webcam view + screen and audio recording). 

It allows webmasters to go in-depth with the criteria and traits they seek from participants. Userlytics also enables users to create screening questions that disqualify users from the study. The platform goes deep into segmentation for the studies, thus rendering more accurate results.

Furthermore, Userlytics also offers advanced quantitative tools like card sorting and tree testing that can be integrated with one’s qualitative usability tests.

4. Crazy Egg

Crazyegg is a click-based user experience tool with features like ‘Heatmap’ logs that show where each visitor clicked on your webpage. Or even a ‘scroll map’ that shows how far down the page each visitor usually scrolls. 

It also offers an overlay feature that breaks down the number of clicks on each page element. It also presents detailed insights about visitor sources, search terms, and other components.

5. Optimizely

Optimizely is a popular A/B Testing platform that enables users to track visits and conversions. The tool boasts a range of features like mobile website testing, geotargeting, cross-browser testing, visitor segmentation and multivariate testing.

Optimizely

6. Qualaroo

The Qualaroo usability testing tool prompts users to answer targeted questions and surveys in real-time on test sites. This unique tool has the ability to integrate with other tools, like Salesforce and Marketo. 

It also has exit surveys that webmasters can utilise to discover why site visitors don’t convert. Additionally, it has a ‘Skip logic’ capability to analyse visitors’ responses before targeting them with a custom follow-up question.

7. Feedback Army 

This user testing tool utilises Amazon’s Mechanical Turk service to solicit user responses. Its three-step process is quicker than setting up your own Mechanical Turk test. 

It basically involves:

  • Uploading the URL of the page that you seek to test.
  • Setting three to six questions about the page.
  • Waiting for your ten responses to arrive.

8. Userzoom Go

Userzoom Go is a comprehensive user research system that features a dedicated testing platform, recruiting services, and automated reporting capabilities. 

Essentially, it focuses on enabling study organisers to focus on the tasks and questions while it handles it recruits, sets up, and carries out the tests. 

userzoom go

9. Loop11

The Loop11  usability testing tool focuses on unmoderated remote tests. It offers a set of tools like A/B testing, testing of live websites or prototypes, heat maps and clickstream analysis. 

Because Loop11 doesn’t maintain an active participant database, users are required to bring their own participants.

10. HotJar

HotJar offers heatmaps as its key feature. Its data visualisation has earned it positive reviews with users as it presents data in an easy manner to draw actionable insights. 

Furthermore, its heatmaps, move maps and scroll maps can be easily downloaded with one click. It also offers a recording feature for users to see the exact behaviour of participants. Additionally, it provides an option to create surveys easily.

The article is a part of our comprehensive guide on “Usability testing”.

6 Types of Survey Respondents to Be Cautious About

6 Types of Survey Respondents to Be Cautious About

Jan 11, 2022 | Respondent Recruitment, User Research | 0 comments

A survey is essentially a research methodology employed to collect data from a predefined group of respondents to gain insights into specific topics of interest.

For the most part, survey research aims to assess a specific market or user needs or determine whether or not particular business objectives have been met. This helps to establish baselines against which future comparisons can be made.

However, a vital ingredient that is central to the success of any survey research is appropriate respondents recruitment in order to get representative respondents for a target market. 

types of Survey Respondents

Who are survey respondents?

A survey respondent is essentially any individual who answers a survey distributed via email, mobile apps, websites, QR codes, or social media. 

Survey respondents are typically sought from samples of the population. However, it’s important to point out that surveys only provide estimates for the true population. Not exact measurements!

Why are survey respondents needed for user research?

Survey respondents can make or break a user research project. The data provided from respondents generate a number of variables that can be actionably studied. For example, the right sample size allows researchers to make fair generalisations. However, a small sample size limits their ability to identify patterns and spot trends.

As a result, finding the right survey respondents is mission-critical to the success of your survey. However, this is sometimes difficult to do. 

Survey respondents to watch out for

Generally speaking, survey respondents come in multiple forms, shapes, and sizes. This means that sometimes there are bad apples, good ones, and some in between. 

Consequently, since survey respondents aren’t one and the same, researchers have to be cautious of how their disparities might impact survey results. Here are some notable respondent types to look out for. 

Speedy respondents

Speedsters move too fast through a survey to provide thoughtful and honest answers. In all honesty, speedsters aren’t really motivated and only aim to complete a survey to receive their incentive. 

Fortunately, parameters around the length of time a respondent is required to spend on a question typically discourage this type of respondent. 

Flatliners

Sometimes called straighteners, such respondents can go one of two ways. Overly positive flatliners frequently select top box answers like “strongly agree”.

On the other hand, negative flatliners typically choose bottom box answers like “strongly disagree.” All in all, flatliners respondents typically have some sort of unwanted acquiescence bias that makes them respond in such a way. 

Cheaters

Cheaters are typically both real respondents and fake ones. Fake ones come in the form of software bots that gain access to online surveys to redeem the rewards. This is without any human having to take the actual survey.

On the other hand, real cheater respondents create multiple accounts to take the same study more than once. Or attempt to take the same study as many times as they possibly can.

To address this, most panels and platforms typically deploy technology parameters to identify and remove bots. Or simply check that respondents aren’t coming from the same IP address in multiple instances. 

Additionally, thoroughly reading through open ends as well as double-checking contact information can help catch cheaters that may have fluked through.

Posers

Posers are the hardest low-quality respondents to identify. Poser respondents, unfortunately, don’t give honest feedback. In some instances, they choose to follow group discussions because of social desirability bias. 

This means they don’t provide their true thoughts or feelings for fear of being different from the crowd. Or even fear of being different from what they assume the survey provider wants to hear.

Professionals

Professionals are survey respondents who are usually categorised as good. These survey takers frequently take different studies and treat them as a job. 

However, sometimes their consistent participation can at times lead to biased results. Especially if they are repeating studies on similar subjects. 

To avoid this, it is important to screen professionals by querying if they have recently taken a survey. Especially if it’s specific to the subject on which you’re conducting research. Then terminate them if they have.

Confused Rule Breakers

Such survey respondents have a difficult time following instructions. Some may intentionally break survey rules, while others could be misinterpreting questions. 

To avoid them, consider performing quality checks by using different screening questions. 

These would include questions that ensure that the respondents belong to the right category and also requires them to read the question before choosing the answer. 

Let us understand this with the following example:

Screening questions can be of two types: Behavioural and industry-specific. 

For the former, the questions include discussing certain behavioural aspects of the respondents (while also ensuring that they read the question and not just answer in yes/no).

exercise

Photo by kike vega on Unsplash

Say, for example, you’re selecting respondents for an exercising app and have multiple questions, some of which could look like these:

 How often do you exercise?

  • Once a week
  • Thrice a week
  • More than 45 minutes every day
  • Rarely

What is your preferred exercise type?

  • Aerobics
  • Swimming
  • Gym 
  • Yoga
  • Running/Jogging
  • None

Such questions naturally help you to eliminate the respondents who choose the answers “rarely”/ “none” as those who would not be using your app. These are, therefore, not the best-qualified respondents to provide accuracy to your surveys. 

Similarly for industry-specific screening, questions ensure that the respondents who are selected fit the requirements of understanding the niche for which the survey is conducted. 

Example: Do you work in any one of the following industries?

  • Teaching
  • Educator
  • Special needs educator
  • Academic writing
  • None of these

Now, for a website/app that’s being tested within the educational niche, the respondents choosing “none of these” are naturally not as qualified as those choosing the other options. 

Screening questions like these ensure that the chosen respondents are aware of your purpose, following the survey instructions and reading the questions before selecting the response. 

Respondent Recruitment Services For Representative User Insights

NX logo
NX logo

Respondent Recruitment Services For Representative User Insights

Key Highlights
  • Dedicated and experienced project manager.
  • Comprehensive briefing call with client to fully understand respondent screener criteria.
  • Screening shortlisted participants and conduct quality check.
  • Scheduling the shortlisted & qualified participants, as well as backups.
  • Liaison with the participants and reminders.
  • Recruitment expert will assist the participants throughout the entire study.
LEARN MORE
CONTACT US

Conclusion 

In successful surveys, choosing the right respondents is critical. This is important to avoid biased opinions that could negatively influence the outcome of the research/study. The nature and accuracy of responses matter.

Remember that even when a study is well written, analysed, and executed, the outcome is only as good as its respondents.

The article is a part of our series on “How to recruit the right respondents for user research?”