What Is Guerrilla Testing and How to Use It?

What Is Guerrilla Testing and How to Use It?

Oct 29, 2021 | UX Guide | 0 comments

More often than not, new products like electronic appliances, websites and mobile apps can cause users frustration as they may be tedious to use, lack flexibility, and take attention away from their core purpose.  Today’s topic focuses on one of the techniques to mitigate such scenarios by employing guerrilla usability testing.

What is Guerrilla Testing?

In essence, guerrilla testing is a usability testing technique that involves gathering user feedback. It is accomplished by taking a product design or prototype to the public and asking random passersby for their thoughts.  Typically, guerrilla testing sessions last for only 10-15 minutes, with a small incentive given to users like a coffee, coupon or cake. Because of its simplicity, new ideas can be quickly tested at a low cost. As such, guerrilla usability testing is considered an inexpensive means of testing mobile apps, product prototypes or websites with real-life users. 

Guerrilla Testing Deliverables

For the most part, guerrilla testing deliverables are typically more qualitative than quantitative as they involve direct assessment of participants. In practice, qualitative approaches tend to query ‘why’ rather than ‘how many’ or ‘how much’.  As a result, guerrilla testing seeks to inform development and design decisions for ongoing projects to identify usability issues, rather than assess the overall usability of an existing product or interface. Thus, guerrilla testing is ideal for:

  • Getting fast baseline measures of an existing product experience.
  • Identifying crucial usability issues early in a product design lifecycle.
  • Testing hypotheses and validating assumptions during design sprints. 
  • Validating tasks that do not necessitate specific knowledge (for example, completing a signup form, or ordering a product in an e-commerce store)

Guerilla Testing vs Usability Testing

Usability testing revolves around testing how easy a particular design is to use with a specific group of representative users. In practice, it traditionally entails observing users’ reactions and behaviours to a website, app or product. The group of users attempt to complete tasks at different stages, from early development until a product’s release. It is carried out in a controlled environment, such as a lab or designated room or online 1-to-1 session.  Guerrilla testing is considered a subset of usability testing and takes a more agile approach to testing a prototype, product or website. High-level feedback is derived to find and fix potential UX issues. Guerrilla testing can also be performed at various stages in the project’s life cycle. 

Benefits of Guerrilla Testing

Guerrilla testing offers numerous advantages to product developers, such as:

  • Quick turnaround as there is no waiting around for recruiters to find people with specific qualifications or attributes. 
  • Inexpensive compared to formal testing as there are no travel costs for users.
  • It enables testers to identify any UX barriers early in the development process.
  • It is iterative and works well with an agile project approach.
  • It provides sufficient or enough insights to inform strategic design decisions. 
  • Guerrilla research is flexible and can be squeezed into nearly every timetable or deadline.
  • It can be utilized to demonstrate the value of user testing/research for stakeholders, especially for those who may struggle to acknowledge the value of usability testing.
  • It is a great way to do ad hoc user research, whether when conducting competitor analysis for similar ideas or practising moderation skills. 
  • It delivers substantial “context of use” observations. 
  • It provides a lot of “usefulness” feedback in real-life situations. 

Shortcomings of Guerrilla Testing

There are, however, some disadvantages that should also be kept in mind:

  • You may not get the right target audience.
  • It could be challenging to record feedback.
  • Sessions are short, so one could lose some of the insights that they would typically get from formal user testing.
  • It may not be really appropriate for all types of websites or mobile apps.
short discussion

Photo by Headway on Unsplash

How to Conduct Guerrilla Testing?

In contrast to recruiting a particular targeted audience to take part in testing sessions, with guerrilla testing, participants are usually approached in public locations and asked to take part. Since there is no formal recruitment or requirement for expensive research facilities, guerrilla testing sessions are fast and easy to set up and can be conducted anywhere like a coffee shop, library, park.  It is recommended to test between 6 -12 users, though this can vary depending on who and where you are testing from. Guerrilla testing sessions are typically short (10–15 minutes) and are structured around specific key research objectives.  If you plan to record the sessions, it is advisable to get participant consent. Each guerrilla testing session can be initiated with the following number of steps:

  • Approach a potential participant. 
  • Politely introduce yourself to potential participants and ask if they would like to partake in the software or product testing session.
  • If they agree, get general information about them.
  • Have them sign a consent form. 
  • Give them a few different scenarios.
  • Carefully observe their interactions.
  • Ask about their experience with the product.
  • Thank and reward them for participating.

Tips for Guerrilla Testing 

Following are some tips to help you achieve your testing objectives:

  • Carefully think about all the critical things people need to be able to achieve while using your product, and write down a shortlist of tasks. For instance, if your product is a mobile app for ordering food, you’ll want to test how people find a particular meal, order multiple meals, or add a meal to their cart.
  • After listing your tasks, prioritise them and decide what to exactly test. Then choose the top 3 tasks, and use them to create scenarios users can easily comprehend.
  • Ensure to always create an elaborate scenario based on each task. In essence, a good scenario describes a problem for participants to solve and is relatable, but does not hint to the participant how to achieve the ultimate goal.
  • Before testing scenarios with test participants, ensure to pre-test them with friends and colleagues to ensure that people can follow them without any confusion. 
  • Always be ethical and transparent.
  • Understand people’s expectations even if your design is not fully complete. For instance, use this opportunity to ask them what they might expect to see after clicking on a certain button/link in a User Interface (UI) flow 
  • Because guerrilla testing is meant to be an ad hoc technique, it does not mean it should be entirely unplanned. So, make sure you carefully plan how much time you’re going to ask of users.
  • Always stay polite and ask users if they are okay with you stealing a little more time than planned. Generally, people won’t be happy if you don’t respect their time, which may lead to biased or even unhelpful responses to your questions. 
  • Avoid always conducting your guerrilla testing in the same places or types of places. Testing at different places will help ensure that you have participants with varied demographics. 

Guerrilla Testing Questions

As we have established, guerrilla testing is a low-cost usability testing method that helps you quickly answer straightforward usability questions. That being said, guerrilla testing questions vastly differ from those employed in more structured and in-depth research methods. Principally, guerrilla testing questions are shallower in their depth of insight, and seek to be informal when engaging users. 

What should you not test?

Guerrilla testing should not be employed in specific scenarios like:

  • When domain-specific knowledge is necessary to use a product (for example, when completing specific use-cases in financial or even medical apps). In practice, you cannot expect random people to have all the required skills to deliver satisfactory technical answers.
  • When a highly specific environment is necessitated to conduct testing (for instance, when testing can be done only in a specific location).

Guerrilla Testing Example

Overall, where you conduct guerrilla tests affects how you perform and document your work. For example, if you’re testing a new mobile app for a supermarket chain, you might go to the store itself and walk down the aisles.  However, if you’re working on a “general” office software suite, you might test it with workers in a different part of an office, etc.  Overall, the idea is to let context drive your decision making. Though public spaces and shopping malls may present some of the best locations for guerrilla testing due to the sheer amount of free foot traffic they receive. Not to mention the relaxed nature of the environment, which can come in handy when approaching strangers. However, with more particular user sets, you can target subjects based on context and demographics. Furthermore, companies can perform remote guerrilla testing by exploiting public forums such as Reddit, Quora or even LinkedIn Groups, by simply writing a simple post describing the intent and the related incentive.

Takeaway

In conclusion, if you’re seeking quick feedback on your prototype or website, then guerrilla usability tests are the way to go. Guerrilla testing allows you to conduct multiple expedited field usability tests during the course of project development to gain actionable insights into where usability barriers could be. However, you have to be considerate of the occasional risk of not getting in front of the right target audience, and sessions being much shorter than with formal usability testing. This might limit your ability to get comprehensive insight.  Nonetheless, guerrilla testing gets you in front of all user types; whether it’s a prospective user, or Peter from down the road, who opens every app under the sun, apart from yours.  Overall, guerrilla testing is a great way to get your foot in the door to demonstrate the value of spending time with users before launching any final product. Reach out to us at Netizen Experience to discuss your usability testing needs.

User Acceptance Testing (UAT) And 10 Best Practices

User Acceptance Testing (UAT) And 10 Best Practices

Oct 7, 2021 | User Acceptance Testing | 0 comments

For the most part, UAT or user acceptance testing is considered a quality assurance step that ensures that all the buttons, visual elements, and attributes work, and there are no technical bugs in the software. 

The main purpose of UAT is to check the software against the business requirements. The ‘user’ in this context is usually referring to the business user or client or person who is familiar with the business requirement. 

It is also common for business users to invest in a specialist testing team to help them conduct the UAT and present the UAT result to them to see if it is in accordance with their expectations for acceptance.

As a matter of fact, it differs from usability testing in that a piece of software can pass all UAT tests but still demonstrate poor usability. 

What is User Acceptance Testing (UAT)?

Generally speaking, User Acceptance Testing (UAT) is where a website or mobile application is iteratively checked to ascertain if it performs the functions it was set out to conduct, according to defined expectations. 

That being said, UAT is typically the last phase of the software testing process before rollout and involves actual software users testing the software to ensure it can handle required tasks in real-world scenarios, according to defined specifications. 

What is the objective of User Acceptance Testing (UAT)?

The primary goal of User Acceptance Tests is to validate that software does what it was supposed to do. However, to achieve this, all functional requirements are typically extensively checked, and use cases are covered while dealing with real scenarios and real data.

So, UAT tests generally revolve around verifying that a system’s behaviour is consistent with its specified requirements to reveal defects within the system. The UAT process begins after acceptance requirements are written and continues through the final stage of testing. 

Overall, UAT testing is based on three elements:

  • Business requirements
  • Business processes
  • User expectations

What is the scope of UAT?

Most UAT operations focus on ensuring that data capture, data processing, data distribution and data archiving activities on a website are fully operational. 

They also ensure all business functions are performed, as per prescribed business rules, and that all business systems connected to the software system in UAT, pass and receive data or control as defined in the requirements specification.

Levels of user acceptance testing

  • Unit testing
  • Integration testing
  • System testing
  • Acceptance testing or UAT.

Types of user acceptance testing techniques

The main types of User Acceptance Testing methods are:

  • Alpha & Beta Testing
  • Black Box Testing
  • Contract Acceptance Testing
  • Regulation Acceptance Testing
  • Operational Acceptance Testing

What are the key deliverables in user acceptance testing?

The key deliverables of the UAT phase are:

  • Test Plan: This mainly outlines the Testing Strategy
  • UAT Test cases: The Test cases help the team effectively test the application in the UAT environment.
  • Test Results and Error Reports: This is essentially a log of all the test cases executed and the actual results.
  • Installation Instructions: This document helps to install the system in a production environment.
  • UAT Sign-off: This ensures that the system, documentation, and training materials have passed all tests within defined acceptable margins.
  • Documentation Materials: Here, tested and updated user documentation and training materials are finalised during UAT.

Stages at which user acceptance testing is performed

UAT is mainly performed at two stages:

  1. Sequential development: This stage employs a sequence of development stages that usually follow a V-shape. UAT is the final test level that tests the completed system with respect to the business requirements.
  2. Iterative Development: This approach entails the design and testing occurring during short sprints. As a result, the system functionality becomes available incrementally at the end of each sprint. Typically, UAT is required before rolling out each sprint.

10 best practices for user acceptance testing 

  1. Set up the right environment: A production-like test environment is imperative as accurate UAT cannot be carried out in generic test environments.

    Ensure that users identified for UAT have proper credentials and data relevant to them to test with. Also extensively run through the test cases to ensure they make sense, data is completely set up, and the system functions as expected.
  2. Identify users:  Generally, user selection is critical to the success of UAT. This means that it is imperative to have those who are familiar with the business requirements for the software that is being developed.

     

    In practice, these users can be experienced or inexperienced, internal or external, with varying job roles. As mentioned earlier, it is also common for businesses to hire a specialist testing team to help them to conduct independent UAT in order to save time.

  3. Identify appropriate test cases and document them: UAT testers require a step-by-step plan for guidance during testing as this will be the first time they’ll see new features and need a roadmap to concentrate their attention in the right places. So, ensure that these steps are clearly laid out and easy to follow.
  4. Carefully plan your tests: Typically, UAT is conducted before the launch of the software. Unfortunately, at this stage, you are already under excessive pressure to meet deadlines. So, performing UAT at this stage could result in the missing of some real-life use cases which are frequent.

     

    While it might seem obvious, designing a clear test acceptance plan is critical during the requirement analysis and design phase as planning reduces pressure to meet deadlines.

  5. Don’t employ the functional testing team as functional testers aren’t meant to perform UAT. This is because functional testers may not test all real-world scenarios, which could lead to end-users discovering issues when the software is in production.

  6. Employ user stories as a way of defining your tests to keep the right level of detail.

     

  7. Do not be biased: Ensure that you give total authority to people who are testing your software product. Don’t interfere or project your thoughts or inherent biases onto them.

     

  8. Always create scenarios based on business requirements.

     

  9. Bug triage and prioritisation: Whenever challenges in UAT are encountered, ensure to have a solid triage plan to address them. It’s a good practice to triage the problem by having the UAT tester show you how it was discovered and extensively document the underworkings of the problem (for example, the browser they were using, their user ID, and what they were trying to accomplish).

     

    In simple terms, the easier you can make it for the development team to recreate and ultimately fix the problem, the better.

    Also, if the UAT tester is stuck, ensure to help them move on with their testing with another approach or change the course of their testing to move away from the problem.  This will probably mean having a standard template to record issues discovered and recap them at the end of the day.

  10. Respect UAT tester feedback: Always solicit UAT testers’’ opinions on the severity of the elements discovered so you can have a basis to discuss any known workarounds. Utilise their thoughts to help prioritise the fixes as problems not immediately addressed can go into one’s backlog for future consideration.

Characteristics of a good user acceptance test

  • They are based on the user roles and business processes that are needed to support the business. 
  • They are written so that different users will execute scenarios slightly differently. 
  • They clearly describe the expected results. 
  • They are typically written in a way that business users can easily understand.
  • They specify the data to be utilised as this is essential to the outcome of the test. 
  • They are verified by the business users before test execution starts.

Key takeaways

As has been noted, UAT is one of the most crucial phases of quality assurance, because when executed correctly, it is highly effective in reducing both time and cost while also increasing customer satisfaction. 

Furthermore, it reduces the probability of development issues being raised, which, in turn, reduces the amount of work required in long-term development and maintenance. 

It’s important to remember that UAT largely depends on the perspective of the business user and isn’t a highly predictable process. So, when conducting user acceptance testing ensure to:

  • Extensively test on multiple devices and operating systems that target users use 
  • Ensure traceability to requirements
  • Test business rules 
  • Document test evidence and record test parameters
  • Generate a test report or summary for sign off