Different Types of Usability Testing Methods

Different Types of Usability Testing Methods

Feb 18, 2022 | User Acceptance Testing, User Testing | 0 comments

Usability testing involves using different techniques to ensure that users of a product can execute intended tasks efficiently, effectively, and satisfactorily. 

In usability testing, prospective users will be recruited to perform tasks typical of a user group in an environment with a website, system, or physical product.

Over time, it has evolved from experimental psychology methods to less controlled and more qualitative tests. 

This article discusses different usability testing methods and their applications.

Quantitative vs Qualitative Usability Testing

The main differences between quantitative and qualitative testing lie in how the data is collected. Generally, usability testing involves two types of data: qualitative and quantitative.

Therefore it is vital to understand the distinctions between these and how to employ each appropriately. The end goal is to gain valuable insights but their approach varies.

With qualitative usability testing approach, data about behaviours and attitudes are directly collected by critically observing what users do and how they react to a product. Qualitative usability data examples range from product reviews, user comments, descriptions of the issues experienced, facial expressions, and preferences, etc.

In comparison, quantitative usability testing approach accumulates data about users’ behaviours and attitudes indirectly. For instance, quantitative data is typically recorded automatically while participants complete the tasks.  Quantitative data typically constitutes statistical data that is quantifiable in numerical terms. For example, how long it took for someone to complete a task, the percentage of a group that clicked a section of a design, etc.

Moderated vs Unmoderated Usability Testing

Moderated usability testing can be administered in-person or remotely by a trained moderator. The moderator introduces the test to participants, responds to their queries, and asks follow-up questions. On the other hand, unmoderated usability testing is performed without direct supervision.

Moderated usability testing typically produces in-depth results because of the direct interaction between moderator and test participants. The moderator is able to probe and follow up with questions to further uncover the underlying motivation or reasons for the participant’s actions.

However, moderated usability testing can be considerably more expensive to organise and run. Conversely, unmoderated usability testing is cheaper. However, the participant may provide superficial or short answers at times and without moderation it can be difficult to get an in-depth understanding of their answers. 

Explorative vs Comparative Usability Testing

Explorative usability testing is open-ended and involves participants being asked to brainstorm, give opinions, and freely express ideas and concepts. 

The information is usually collected in early product development to help researchers pinpoint market gaps, identify potential new features, and iterate new ideas.

Comparative usability testing involves asking users to choose between solutions they prefer to compare a website or app with its primary competitors.

Remote vs In-Person Usability Testing

In remote usability tests, the facilitator interacts with the participants from a different physical location. These are done over the internet through laptop, dekstop, tablet or smartphone. Moderated remote usability testing can be performed using screen-sharing options from meeting tools like Zoom, Skype, etc. For remote unmoderated usability tests, online tools can be used to set up tasks that the participant needs to perform. 

In-person testing is done in the physical presence of the moderator. As compared to remote tests, it is easier to collect and observe more information as the UX researchers can analyze the facial expressions as well as the body language of the participants and how it correlates with their action. 

Also read: 

How To Conduct Usability Testing For Websites?How to Conduct Usability Testing for Mobile Apps?

Usability Testing Methods

Some of the commonly used usability testing methods for websites and applications include:

1. Guerrilla testing 

Guerrilla testing is arguably the simplest usability testing method. Essentially, guerrilla testing is as simple as going to a public place like a café and asking people about their thoughts on a prototype. 

Basically, test participants are chosen randomly and asked to perform quick tasks, often in exchange for a small gift (like a free coffee). It’s a low-cost approach that works best in the early stages of product development.

guerrilla testing

2. Lab usability testing

Lab usability testing needs a trained moderator and a suitable place for testing.

This test approach is suitable when you need in-depth information on how real users interact with a product, and the issues they may face. 

This method enables you to collect comprehensive and qualitative information. However, it can be expensive to organise and execute since it requires a controlled environment, hiring of test participants and trained moderators. 

3. Contextual inquiry

This usability testing method helps a product team obtain information about the user experience from the real users. This method is perfect for attaining rich information about users— for example, their workspace, personal preferences, and habits.

In this approach, users are first asked a specific set of questions about their experience with a product. Subsequently, they are then critically observed and questioned while working within their own environments. For example, a finance clerk is observed on how she use the new accounting software at her office where she usually works. 

4. Card Sorting

Card sorting is a usability testing process where participants demonstrate how they expect a website to look in terms of navigation. This testing method helps UX designers and UX researchers to discover whether their navigational structure matches what users expect. 

5. A/B Testing

This usability testing method typically involves critically comparing two versions of an application or website against each other to assess discrepancies. Usually, product managers use statistical analysis to fully determine which of the two versions works better.

6. Screen Recording

Screen recording type of usability testing requires software that can record the participant’s actions while they are performing a task remotely. The video recording captures their clicks, cursor movements, expressions, or even their voice when they speak aloud. 

Tools like Hotjar or Zoom can be used to record actions like mouse movement, clicks, etc. 

7. 5- Second Test

This usability testing method is used to collect qualitative data on a user’s first impressions when interacting with a website, app, prototype or a wireframe. This test is usually conducted before a product release to understand the reactions of the target audience. One can try this approach on different variations of a design to see which receives a better response. 

The article is a part of our comprehensive guide on “Usability testing”.

User Testing Services in Asia

NX logo
NX logo

User Testing Services in Asia

Key Highlights
  • Provides user testing services for websites, mobile apps & other digital products
  • Recruit users who match the client’s target audience & provide them with tasks to complete
  • Users are asked to think aloud and provide feedback on their experience
  • Helps conduct user testing sessions remotely, using a variety of tools & software
  • Provides a dedicated project manager to oversee the entire process
  • Project manager coordinates the recruitment of users, schedule of testing sessions & the delivery of the results
LEARN MORE
CONTACT US

How to Conduct Usability Testing for Mobile Apps?

How to Conduct Usability Testing for Mobile Apps?

Feb 15, 2022 | User Acceptance Testing | 0 comments

During usability testing, the moderator typically collect qualitative data like behavioural observations and participant comments. UX Researchers would also sometimes collect quantitative data like task times and success rates. 

By critically analysing this data, the UX researcher seek to identify usability problems within a mobile application or website. 

In today’s blog post, we shall delve into usability testing for mobile applications and its underworkings.

What is mobile usability testing? 

Mobile usability testing is essentially usability testing performed on mobile products and applications running on mobile platforms. 

This usability testing exercise attempts to determine how end users might perceive a mobile app or product, either in a usability laboratory or in a field setting.

Is mobile app usability testing important? 

Mobile app usability tests seek to observe test-subject users while utilising a specific app. The core purpose is to measure the application’s user-friendliness to better support a brand’s key commercial objectives.

Mobile usability testing also ensures an app adds value to a business, thus meeting the expectations of the final users. 

Generally, ensuring good usability for mobile apps helps improve customer satisfaction, decrease time spent on customer support, and increase overall sales and revenue.

Furthermore, a well-tested and improved application attracts visitors’ attention and helps set up trust and cooperation between the app owner and the app’s potential users.

usability testing mobile app

How to run effective usability testing for mobile apps

Step 1: Define objectives

Before commencing any usability testing exercise, you should set your objectives and goals straight. In essence, you should clearly define the questions you want to answer with the usability test. Or even clarify the hypothesis you want to test with the usability test.

As you define the test’s objectives, ensure you explore these important areas:

  • The app’s roadmap (for example if something is important to be tested or it will already be removed in the next update)
  • The potential impact of the test
  • Existing users and markets for whom the app is targeted
  • The app’s existing competitors
  • Timing and scope

Step 2: Design the tasks

Once the test objectives have been defined, you then need to set the tasks of the usability test. Tasks should be one sentence long and consist of the interactions to be performed by the test users, for example:

  • Register an account
  • Upload a photo
  • Accept a friend request
  • Sign into an account

However, rather than directly asking the test user to execute a task, tasks should be converted into task scenarios. These provide more context to participants about why they are doing the task. And thus, facilitate more natural interactions similar to what an ordinary user will perform with your app.

Consequently, task scenarios that are defined should always be:

  • Realistic, actionable and without any obvious clues about how to perform the steps.
  • Sequentially ordered to ensure a smooth flow of the test session.
  • Tied to one or more objectives.

 Step 3:  Prepare the usability test documents

When conducting usability testing for mobile apps, there are several documents you typically require. For example: 

  • Consent forms (for minors and even adults)
  • Post-test questionnaire 

Ensure these documents are well prepared to suit your user group and their nuances.

respondent signing usability test consent document

Step 4: Prepare the test participants

Mobile usability testing typically involves real users undertaking realistic tasks that the mobile app is intended to achieve. 

As you prepare to test with real users, there are several considerations one can take when vetting participants: 

  • Choose users who are a fair representation of the target audience.  
  • Users own a mobile device with the exact operating system (including the appropriate version/s) that is being targeted.
  • Participants must be available at the time, place, frequency of the intended usability tests.
  • Users must agree to the compensation terms that you’re offering (if any)
  • Participants must be ready to sign a usability test participation consent form

Step 5: Choose a mobile application usability testing methodology

Principally, there are two main methods for conducting usability testing of mobile applications. These are:

  1. Laboratory-based usability testing
  2. Remote usability testing

In this step, weigh the pros and cons of each approach and then pick one that will fit your needs. 

Step 6: Reporting the results of the usability test

After collecting your usability test data, the next step is to compile, organise and analyse it to draw meaningful conclusions. 

The data can be split between quantitative and quantitative data. For instance, quantitative data encompass completion rates, task times, success rates, satisfaction ratings and error rates. 

Or qualitative data like problems experienced, answers provided in the questionnaire, post-test interviews and debriefing sessions.

This step basically involves careful analysing notes on recordings, transcripts, and other information you might have gathered. It then moves on to presenting it in a way that delivers actionable recommendations.

Conclusion

Usability testing is vital to the success of mobile apps. Mobile apps that users perceive as easy to learn, user-friendly and less time-consuming tend to be more profitable and popular. 

Furthermore, usability testing should occur on a regular and scheduled basis, especially when introducing new design features, or updates to improve functionality.

The article is a part of our comprehensive guide on “Usability testing”.

Field Studies vs Usability Testing: Which Is Better?

Field Studies vs Usability Testing: Which Is Better?

Feb 14, 2022 | User Research, User Acceptance Testing, User Testing | 0 comments

Usability is essentially the measurement of a product’s usefulness from the users’ perspective. In principle, usability can be segmented into three fundamental elements: efficiency, effectiveness, and user satisfaction. 

As a research methodology, the lack of usability testing during the development of a product often leads to user dissatisfaction and rejection. Another research methodology that serves a similar purpose of ensuring product utility is a field study. 

What is the difference between field studies and usability testing?

Field study encompasses all studies of users in their natural environments (usually conducted in the user’s context and location as opposed to your office/lab). 

For example, homes, workplaces, neighbourhoods, parks, streets, and shops. Essentially, localities and areas in which one’s product might eventually be utilised are the best places to conduct these studies. 

On the other hand, usability testing evaluates a product by directly testing it with representative users. 

During a usability test, participants attempt to complete typical tasks while testers/observers watch, listen and take notes. 

Overall, the core goal of usability testing is to identify usability problems, collect qualitative and quantitative data to determine the participant’s satisfaction with the product.

conducting remote online usability testing on laptop

When to choose a field study? 

Field studies deliver a complete, unbiased picture of what prospective users actually do with a product. They provide context, especially if a product is designed to function in a particular context. 

In such instances, conventional lab testing might not give you accurate results. So, when your usability research cannot fit in a lab, you may need to carry these out. 

Furthermore, field study is well suited for practical products. For instance, if a device is engineered to function as a core component of the navigational system of an oil tanker. Then visiting the oil tanker for final testing makes sense. 

Some of the examples of field studies include user’s office/home visit, ethnographic research, and contextual inquiry. 

How to conduct a field study? 

1. Preliminary planning 

This period involves working with participants. You don’t necessarily require a specific set of research questions but defined study topics. Field studies are strictly observational, as researchers aim to be as unobtrusive as possible. 

2. Direct observation

This part of a field study involves watching users critically observe how they behave and why. Ideally, the test subjects do not care that you’re watching, and act exactly as if you are not there. 

3. Participant observation

Here, the researcher can join the group of people being studied and records data as field notes or diary entries, after finishing observations for the day.

Benefits of field studies in UX design

Field studies enable you to understand your users in-depth, under realistic conditions. Field studies also allow you to discover social defects and understand environmental factors before releasing products.

Generally, field study advantages can be summarised as:

  1. It yields highly detailed data.
  2. It emphasises the role and relevance of social context.
  3. It can help uncover social facts that may not be immediately obvious or that research participants may be unaware of.

In person field work study usability testing on mobile app

Potential pitfalls of field studies in UX design 

The key disadvantage of field studies is their expense. Furthermore, most field studies can’t usually be automated or sped up with technology. This is because they typically rely on old-fashioned theories of patience and observation. 

Additionally, if your study design requires a large number of observers, then a field study might not be ideal. Similarly, if your product is to be used in rare, unpredictable circumstances (for example, first-responder mobilisation after an earthquake), then you can’t do a field study.

Further, if your study involves collecting sensitive, confidential information, you might require the more controllable circumstances of a user research lab.

When to choose usability testing?

Usability testing is ideal when: 

  • Seeking to uncover any issues within your design, workflow, or process.
  • Validating if a design works for participants – especially if there is debate on a project team about how something is ‘utilised by users’.
  • Seeking insights into both good and poor issues with an interface.
  • Seeking different perspectives and mental models on an interface.

You can choose to conduct usability testing:

  • Before significant design decisions are made.
  • In high-risk, low-certainty situations.
  • When it’s time to evaluate and iterate.
  • After product launch.

Benefits of usability testing in UX Design

  • Improved user experience and the product utility.
  • Discovery of hidden usability issues.
  • It ensures that the application’s functionality matches the requirements.
  • It identifies changes required to improve user performance and satisfaction.
  • Helps to analyse product performance to determine if it meets the defined usability objectives.

Disadvantages of usability testing in UX design

  • There is sometimes uncertainty about what to test
  • There can be testing fatigue
  • There is sometimes too much feedback gathered

Field Studies vs Usability Testing Cost

Field studies are expensive and can be highly time-consuming. This is so because of the need to travel, the number of hours researchers are required to commit, and the complex analysis that open-ended, unstructured research dictates.

On the other hand, usability tests are relatively inexpensive and easy to conduct, especially if conducted remotely. However, some of these can only be conducted in a specially designed laboratory, enabling facilitators to interact with, and observe users. Such tests can involve some cost, but they will still be cheaper than field testing.

User Testing Services in Asia

NX logo
NX logo

User Testing Services in Asia

Key Highlights
  • Provides user testing services for websites, mobile apps & other digital products
  • Recruit users who match the client’s target audience & provide them with tasks to complete
  • Users are asked to think aloud and provide feedback on their experience
  • Helps conduct user testing sessions remotely, using a variety of tools & software
  • Provides a dedicated project manager to oversee the entire process
  • Project manager coordinates the recruitment of users, schedule of testing sessions & the delivery of the results
LEARN MORE
CONTACT US

Conclusion

In summary, usability testing is an inexpensive means of gathering valuable feedback from representative users. 

In contrast, field studies allow user researchers to gain first-hand experience and knowledge about the users and the processes they study.

The article is a part of our comprehensive guide on “Usability testing”.

10 Usability Testing Tools to Try in 2022

10 Usability Testing Tools to Try in 2022

Jan 31, 2022 | User Acceptance Testing, User Testing | 0 comments

Choosing an appropriate usability testing tool is vital to executing an effective user experience. Typically, the choice of tool depends on the type of usability testing method you’ll employ.

It is true that you can run usability testing without a specialised tool, for example, when doing qualitative, in-person usability testing. However, if you are planning remote usability testing sessions, or seeking to collect quantitative data, then a dedicated tool is necessary.

What are usability testing tools?

Usability testing tools are dedicated solutions that essentially streamline usability testing activities. These solutions can be utilised to gather actionable insights into how prospective users might perceive and experience your product. 

For example, they can be used to put a prototype or website in front of real users. Thus, enabling website owners to analyse how users accomplish a given task or to identify possible frustration points. 

Read our article on “How to conduct usability testing for websites?”

Generally, usability testing tools smoothen the process of accessing and tracking users’ feedback. They also help with the analysis of that feedback to enable product owners to make data-informed decisions and deliver the best user experience.

Top 10 Usability Testing Tools 

1. Maze

Maze is a rapid user testing platform for collecting both qualitative and quantitative usability data. This platform seamlessly integrates directly with Adobe XD, Marvel, Sketch Figma, InVision. 

Furthermore, Maze allows users to create and run in-depth usability tests, then share those tests with testers via a link. Maze includes varied features like task analysis, guerrilla testing, multiple path analysis, heatmaps, A/B testing, and wireframe testing. 

Maze also generates an instant usability test report for each test that users can share with anyone via a link. 

This reporting functionality presents usability testing results like completion rates, misclick rates, and time spent. Additionally, Maze allows users to run research surveys and test their information architecture using Card Sorts and Tree Tests.

2. Lookback

This is a user experience (UX) screen recording tool for UX designers and product managers who seek to analyse how users interact with their applications. 

Lookback’s testing capabilities enable designers to view what users see and get their reactions in real-time. This is either recorded or in-person. 

For instance, if you set up a remote test, participants shall receive a link to download your app. Then they get started with a live session or self-test, and you can even communicate directly with them. Subsequently, user recordings automatically appear in a dashboard where one can organise them into groups and create highlights. 

Lookback

3. Userlytics

As one of the most popular usability testing tools in the market, Userlytics a UX research service that offers picture-in-picture user recording (webcam view + screen and audio recording). 

It allows webmasters to go in-depth with the criteria and traits they seek from participants. Userlytics also enables users to create screening questions that disqualify users from the study. The platform goes deep into segmentation for the studies, thus rendering more accurate results.

Furthermore, Userlytics also offers advanced quantitative tools like card sorting and tree testing that can be integrated with one’s qualitative usability tests.

4. Crazy Egg

Crazyegg is a click-based user experience tool with features like ‘Heatmap’ logs that show where each visitor clicked on your webpage. Or even a ‘scroll map’ that shows how far down the page each visitor usually scrolls. 

It also offers an overlay feature that breaks down the number of clicks on each page element. It also presents detailed insights about visitor sources, search terms, and other components.

5. Optimizely

Optimizely is a popular A/B Testing platform that enables users to track visits and conversions. The tool boasts a range of features like mobile website testing, geotargeting, cross-browser testing, visitor segmentation and multivariate testing.

Optimizely

6. Qualaroo

The Qualaroo usability testing tool prompts users to answer targeted questions and surveys in real-time on test sites. This unique tool has the ability to integrate with other tools, like Salesforce and Marketo. 

It also has exit surveys that webmasters can utilise to discover why site visitors don’t convert. Additionally, it has a ‘Skip logic’ capability to analyse visitors’ responses before targeting them with a custom follow-up question.

7. Feedback Army 

This user testing tool utilises Amazon’s Mechanical Turk service to solicit user responses. Its three-step process is quicker than setting up your own Mechanical Turk test. 

It basically involves:

  • Uploading the URL of the page that you seek to test.
  • Setting three to six questions about the page.
  • Waiting for your ten responses to arrive.

8. Userzoom Go

Userzoom Go is a comprehensive user research system that features a dedicated testing platform, recruiting services, and automated reporting capabilities. 

Essentially, it focuses on enabling study organisers to focus on the tasks and questions while it handles it recruits, sets up, and carries out the tests. 

userzoom go

9. Loop11

The Loop11  usability testing tool focuses on unmoderated remote tests. It offers a set of tools like A/B testing, testing of live websites or prototypes, heat maps and clickstream analysis. 

Because Loop11 doesn’t maintain an active participant database, users are required to bring their own participants.

10. HotJar

HotJar offers heatmaps as its key feature. Its data visualisation has earned it positive reviews with users as it presents data in an easy manner to draw actionable insights. 

Furthermore, its heatmaps, move maps and scroll maps can be easily downloaded with one click. It also offers a recording feature for users to see the exact behaviour of participants. Additionally, it provides an option to create surveys easily.

The article is a part of our comprehensive guide on “Usability testing”.