6 Types of Survey Respondents to Be Cautious About

Jan 11, 2022 | User Research

A survey is essentially a research methodology employed to collect data from a predefined group of respondents to gain insights into specific topics of interest.

For the most part, survey research aims to assess a specific market or user needs or determine whether or not particular business objectives have been met. This helps to establish baselines against which future comparisons can be made.

However, a vital ingredient that is central to the success of any survey research is appropriate respondents recruitment in order to get representative respondents for a target market. 

types of Survey Respondents

Who are survey respondents?

A survey respondent is essentially any individual who answers a survey distributed via email, mobile apps, websites, QR codes, or social media. 

Survey respondents are typically sought from samples of the population. However, it’s important to point out that surveys only provide estimates for the true population. Not exact measurements!

Why are survey respondents needed for user research?

Survey respondents can make or break a user research project. The data provided from respondents generate a number of variables that can be actionably studied. For example, the right sample size allows researchers to make fair generalisations. However, a small sample size limits their ability to identify patterns and spot trends.

As a result, finding the right survey respondents is mission-critical to the success of your survey. However, this is sometimes difficult to do. 

Survey respondents to watch out for

Generally speaking, survey respondents come in multiple forms, shapes, and sizes. This means that sometimes there are bad apples, good ones, and some in between. 

Consequently, since survey respondents aren’t one and the same, researchers have to be cautious of how their disparities might impact survey results. Here are some notable respondent types to look out for. 

Speedy respondents

Speedsters move too fast through a survey to provide thoughtful and honest answers. In all honesty, speedsters aren’t really motivated and only aim to complete a survey to receive their incentive. 

Fortunately, parameters around the length of time a respondent is required to spend on a question typically discourage this type of respondent. 

Flatliners

Sometimes called straighteners, such respondents can go one of two ways. Overly positive flatliners frequently select top box answers like “strongly agree”.

On the other hand, negative flatliners typically choose bottom box answers like “strongly disagree.” All in all, flatliners respondents typically have some sort of unwanted acquiescence bias that makes them respond in such a way. 

Cheaters

Cheaters are typically both real respondents and fake ones. Fake ones come in the form of software bots that gain access to online surveys to redeem the rewards. This is without any human having to take the actual survey.

On the other hand, real cheater respondents create multiple accounts to take the same study more than once. Or attempt to take the same study as many times as they possibly can.

To address this, most panels and platforms typically deploy technology parameters to identify and remove bots. Or simply check that respondents aren’t coming from the same IP address in multiple instances. 

Additionally, thoroughly reading through open ends as well as double-checking contact information can help catch cheaters that may have fluked through.

Posers

Posers are the hardest low-quality respondents to identify. Poser respondents, unfortunately, don’t give honest feedback. In some instances, they choose to follow group discussions because of social desirability bias. 

This means they don’t provide their true thoughts or feelings for fear of being different from the crowd. Or even fear of being different from what they assume the survey provider wants to hear.

Professionals

Professionals are survey respondents who are usually categorised as good. These survey takers frequently take different studies and treat them as a job. 

However, sometimes their consistent participation can at times lead to biased results. Especially if they are repeating studies on similar subjects. 

To avoid this, it is important to screen professionals by querying if they have recently taken a survey. Especially if it’s specific to the subject on which you’re conducting research. Then terminate them if they have.

Confused Rule Breakers

Such survey respondents have a difficult time following instructions. Some may intentionally break survey rules, while others could be misinterpreting questions. 

To avoid them, consider performing quality checks by using different screening questions. 

These would include questions that ensure that the respondents belong to the right category and also requires them to read the question before choosing the answer. 

Let us understand this with the following example:

Screening questions can be of two types: Behavioural and industry-specific

For the former, the questions include discussing certain behavioural aspects of the respondents (while also ensuring that they read the question and not just answer in yes/no).

exercise

Photo by kike vega on Unsplash

Say, for example, you’re selecting respondents for an exercising app and have multiple questions, some of which could look like these:

 How often do you exercise?

  • Once a week
  • Thrice a week
  • More than 45 minutes every day
  • Rarely

What is your preferred exercise type?

  • Aerobics
  • Swimming
  • Gym 
  • Yoga
  • Running/Jogging
  • None

Such questions naturally help you to eliminate the respondents who choose the answers “rarely”/ “none” as those who would not be using your app. These are, therefore, not the best-qualified respondents to provide accuracy to your surveys. 

Similarly for industry-specific screening, questions ensure that the respondents who are selected fit the requirements of understanding the niche for which the survey is conducted. 

Example: Do you work in any one of the following industries?

  • Teaching
  • Educator
  • Special needs educator
  • Academic writing
  • None of these

Now, for a website/app that’s being tested within the educational niche, the respondents choosing “none of these” are naturally not as qualified as those choosing the other options. 

Screening questions like these ensure that the chosen respondents are aware of your purpose, following the survey instructions and reading the questions before selecting the response. 

Conclusion 

In successful surveys, choosing the right respondents is critical. This is important to avoid biased opinions that could negatively influence the outcome of the research/study. The nature and accuracy of responses matter.

Remember that even when a study is well written, analysed, and executed, the outcome is only as good as its respondents.

The article is a part of our series on “How to recruit the right respondents for user research?

Recent Articles

Evaluative Research: Definition, Methods, and Types of Application

Research is mission-critical to solving navigation challenges in the UX and product design world. In fact, user research helps UX designers to better understand the users beyond their own internal assumptions.  Therefore, enabling designers to put themselves in users’...

Guide to Creating Using Personas for UX

A great user experience is paramount to any mobile application.  After all, nobody likes an application that is horrible to navigate, right? A deep understanding of one’s target audience is mission-critical to creating an exceptional user experience (UX).  One...

Information Architecture in UX – A Complete Guide

Due to the proliferation of mobile devices, information available to humans keeps growing on a daily basis. The more data we have, the harder it is to sift through to find what you need.  Fortunately, a well-thought-out information architecture (IA) can help users...

Remote UX Workshops: 10 Tips To Be Successful

With a large majority of people working from home since 2021, we have had to find suitable remote solutions. Whether it's for meetings, projects, user research or just a quick catch-up. UX Workshops are now having to be done remotely as well, and design thinking...

Why Personalisation Matters in User Experience?

User experience (UX) activities primarily revolve around enhancing user satisfaction by improving the usability and accessibility for users interacting with physical or digital products.  These activities usually include user research, sketching, interaction...