Different Types of Usability Testing Methods

Feb 18, 2022 | User Acceptance Testing, User Testing

Usability testing involves different techniques to ensure that users of a system can execute intended tasks efficiently, effectively, and satisfactorily. 

In usability testing, prospective users perform tasks typical of a user group in an ordinary environment with a website, system, or physical product.

Over time, it has evolved from experimental psychology methods to less controlled and more qualitative tests. 

Quantitative vs Qualitative Usability Testing

The main differences between quantitative and qualitative testing lie in how the data is collected. Generally, usability testing involves two types of data: qualitative and quantitative.

It is, therefore, vital to understand the distinctions between the two and how to employ each appropriately. The end goal is to gain valuable insights; however, their approach varies.

With qualitative testing, data about behaviours and attitudes are directly collected by critically observing what users do and how they react to a product. 

In comparison, quantitative testing accumulates data about users’ behaviours and attitudes indirectly. For instance, quantitative data is typically recorded automatically while participants complete the tasks.

Qualitative usability data examples range from product reviews, user comments, descriptions of the issues experienced, facial expressions, and preferences, etc.

On the other hand, quantitative data typically constitutes statistical data that is quantifiable in numerical terms. For example, how long it took for someone to complete a task, the percentage of a group that clicked a section of a design, etc.

Moderated vs Unmoderated Usability Testing

Moderated usability testing is administered in-person or remotely by a trained moderator. The moderator introduces the test to participants, responds to their queries, and asks follow-up questions. On the other hand, unmoderated usability testing is performed without direct supervision.

Moderated testing typically produces in-depth results because of the direct interaction between moderator and test participants. Moderator is able to probe and follow up with questions to further uncover the underlying motivation or reasons for the participant’s actions.

However, it can be considerably more expensive to organise and run. Conversely, unmoderated testing is cheaper. However, participant answers can provide superficial answers at times. 

Explorative vs Comparative Usability Testing

Explorative usability testing is open-ended and involves participants being asked to brainstorm, give opinions, and freely express ideas and concepts. 

The information is usually collected in early product development to help researchers pinpoint market gaps, identify potential new features, and iterate new ideas.

Comparative usability testing involves asking users to choose between solutions they prefer to compare a website or app with its primary competitors.

Usability Testing Methods

1. Guerrilla testing 

Guerrilla testing is arguably the simplest usability testing method. Essentially, guerrilla testing is as simple as going to a public place like a café and asking people about their thoughts on a prototype. 

Basically, test participants are chosen randomly and asked to perform quick tasks, often in exchange for a small gift (like a free coffee). It’s a low-cost approach that works best in the early stages of product development.

guerrilla testing

2. Lab usability testing

Lab usability testing needs a trained moderator and a suitable place for testing.

This test approach is suitable when you need in-depth information on how real users interact with a product, and the issues they may face. 

This method enables you to collect comprehensive and qualitative information. However, it can be expensive to organise and execute since it requires a controlled environment, hiring of test participants and trained moderators. 

3. Contextual inquiry

This usability testing method helps a product team obtain information about the user experience from the real users. This method is perfect for attaining rich information about users— for example, their workspace, personal preferences, and habits.

In this approach, users are first asked a specific set of questions about their experience with a product. Subsequently, they are then critically observed and questioned while working within their own environments. For example, a finance clerk is observed on how she use the new accounting software at her office where she usually works. 

4. Online or remote usability testing

Usability testing is carried out with participant by connecting with them over the internet. This method of usability testing is able to save time and lessen the need for respondents travelling in person to the usability testing lab. It also provides the added advantage which the participant can join the usability test within the comfort of their home. Screen sharing is usually done online so that the UX researcher can see what the respondent is seeing during the usability test.

5. Card Sorting

Card sorting is a usability testing process where participants demonstrate how they expect a website to look in terms of navigation. This testing method helps UX designers and UX researchers to discover whether their navigational structure matches what users expect. 

7. A/B Testing

This usability testing method typically involves critically comparing two versions of an application or website against each other to assess discrepancies. Usually, product managers use statistical analysis to fully determine which of the two versions works better.

The article is a part of our comprehensive guide on “Usability testing”.

Recent Articles

How To Use Affinity Mapping For UX Research?

Sometimes, cross-functional UX teams are required to analyse substantial amounts of research data and synthesise their findings to achieve the desired user experience outcome. One tool that helps facilitate this process is affinity mapping.  This article will delve...

What is a UX Case Study? (+5 examples)

UX designers should be able to articulate their ideas in a clear and concise manner. Creating a comprehensive UX case study for a UX design portfolio is one intuitive way to stand out from the crowd. It is straightforward to create one, but there are specific factors...

Top 12 Malaysia eCommerce Websites

eCommerce (electronic commerce) is essentially a series of activities that revolve around the buying or selling of products or services over the Internet. It is underpinned by technology-driven activities like electronic funds transfer, data interchange, transactions...

What is Data Analytics – Explained

We are in the 21st century, where data analytics is one of the most popular terminologies and is used nonchalantly across industries.  It is because a lot of decision-making has shifted from gut-based and impromptu to data-backed. It not only includes designing but...