6 Types of Survey Respondents to Be Cautious About

Jan 11, 2022 | User Research, Financial Services, Respondent Recruitment

A survey is essentially a research methodology employed to collect data from a predefined group of respondents to gain insights into specific topics of interest.

For the most part, survey research aims to assess a specific market or user needs or determine whether or not particular business objectives have been met. This helps to establish baselines against which future comparisons can be made.

However, a vital ingredient that is central to the success of any survey research is appropriate respondents recruitment in order to get representative respondents for a target market. 

types of Survey Respondents

Who are survey respondents?

A survey respondent is essentially any individual who answers a survey distributed via email, mobile apps, websites, QR codes, or social media. 

Survey respondents are typically sought from samples of the population. However, it’s important to point out that surveys only provide estimates for the true population. Not exact measurements!

Why are survey respondents needed for user research?

Survey respondents can make or break a user research project. The data provided from respondents generate a number of variables that can be actionably studied. For example, the right sample size allows researchers to make fair generalisations. However, a small sample size limits their ability to identify patterns and spot trends.

As a result, finding the right survey respondents is mission-critical to the success of your survey. However, this is sometimes difficult to do. 

Survey respondents to watch out for

Generally speaking, survey respondents come in multiple forms, shapes, and sizes. This means that sometimes there are bad apples, good ones, and some in between. 

Consequently, since survey respondents aren’t one and the same, researchers have to be cautious of how their disparities might impact survey results. Here are some notable respondent types to look out for. 

Speedy respondents

Speedsters move too fast through a survey to provide thoughtful and honest answers. In all honesty, speedsters aren’t really motivated and only aim to complete a survey to receive their incentive. 

Fortunately, parameters around the length of time a respondent is required to spend on a question typically discourage this type of respondent. 

Flatliners

Sometimes called straighteners, such respondents can go one of two ways. Overly positive flatliners frequently select top box answers like “strongly agree”.

On the other hand, negative flatliners typically choose bottom box answers like “strongly disagree.” All in all, flatliners respondents typically have some sort of unwanted acquiescence bias that makes them respond in such a way. 

Cheaters

Cheaters are typically both real respondents and fake ones. Fake ones come in the form of software bots that gain access to online surveys to redeem the rewards. This is without any human having to take the actual survey.

On the other hand, real cheater respondents create multiple accounts to take the same study more than once. Or attempt to take the same study as many times as they possibly can.

To address this, most panels and platforms typically deploy technology parameters to identify and remove bots. Or simply check that respondents aren’t coming from the same IP address in multiple instances. 

Additionally, thoroughly reading through open ends as well as double-checking contact information can help catch cheaters that may have fluked through.

Posers

Posers are the hardest low-quality respondents to identify. Poser respondents, unfortunately, don’t give honest feedback. In some instances, they choose to follow group discussions because of social desirability bias. 

This means they don’t provide their true thoughts or feelings for fear of being different from the crowd. Or even fear of being different from what they assume the survey provider wants to hear.

Professionals

Professionals are survey respondents who are usually categorised as good. These survey takers frequently take different studies and treat them as a job. 

However, sometimes their consistent participation can at times lead to biased results. Especially if they are repeating studies on similar subjects. 

To avoid this, it is important to screen professionals by querying if they have recently taken a survey. Especially if it’s specific to the subject on which you’re conducting research. Then terminate them if they have.

Confused Rule Breakers

Such survey respondents have a difficult time following instructions. Some may intentionally break survey rules, while others could be misinterpreting questions. 

To avoid them, consider performing quality checks by using different screening questions. 

These would include questions that ensure that the respondents belong to the right category and also requires them to read the question before choosing the answer. 

Let us understand this with the following example:

Screening questions can be of two types: Behavioural and industry-specific

For the former, the questions include discussing certain behavioural aspects of the respondents (while also ensuring that they read the question and not just answer in yes/no).

exercise

Photo by kike vega on Unsplash

Say, for example, you’re selecting respondents for an exercising app and have multiple questions, some of which could look like these:

 How often do you exercise?

  • Once a week
  • Thrice a week
  • More than 45 minutes every day
  • Rarely

What is your preferred exercise type?

  • Aerobics
  • Swimming
  • Gym 
  • Yoga
  • Running/Jogging
  • None

Such questions naturally help you to eliminate the respondents who choose the answers “rarely”/ “none” as those who would not be using your app. These are, therefore, not the best-qualified respondents to provide accuracy to your surveys. 

Similarly for industry-specific screening, questions ensure that the respondents who are selected fit the requirements of understanding the niche for which the survey is conducted. 

Example: Do you work in any one of the following industries?

  • Teaching
  • Educator
  • Special needs educator
  • Academic writing
  • None of these

Now, for a website/app that’s being tested within the educational niche, the respondents choosing “none of these” are naturally not as qualified as those choosing the other options. 

Screening questions like these ensure that the chosen respondents are aware of your purpose, following the survey instructions and reading the questions before selecting the response. 

Respondent Recruitment Services For Representative User Insights

NX logo
NX logo

Respondent Recruitment Services For Representative User Insights

Key Highlights
  • Dedicated and experienced project manager.
  • Comprehensive briefing call with client to fully understand respondent screener criteria.
  • Screening shortlisted participants and conduct quality check.
  • Scheduling the shortlisted & qualified participants, as well as backups.
  • Liaison with the participants and reminders.
  • Recruitment expert will assist the participants throughout the entire study.

Conclusion 

In successful surveys, choosing the right respondents is critical. This is important to avoid biased opinions that could negatively influence the outcome of the research/study. The nature and accuracy of responses matter.

Remember that even when a study is well written, analysed, and executed, the outcome is only as good as its respondents.

The article is a part of our series on “How to recruit the right respondents for user research?

Recent Articles

Qualitative Usability Testing Tips

Usability testing is a crucial part of the UX design process that helps businesses understand how users interact with their products and identify areas for improvement. One effective approach to usability testing is qualitative testing, which focuses on gathering...

Generative AI: What Is It and How Can It Help With User Research?

Understanding user behaviour and preferences is of the utmost importance in the ever-changing field of user experience design. At this stage, user research becomes relevant, providing valuable insights that can inform design decisions. The problem with tried-and-true...

Why Are Open-Ended Questions Important In Qualitative Research?

Qualitative research is crucial in understanding the complexities of human behaviour, experiences, and perspectives. It allows researchers to explore the richness and depth of individuals' thoughts, feelings, decision making process and motivations. One of the...

Generative Research: All You Need To Know

Human-centred research should always be an integral component of any UI/UX design process than isolated from it. Furthermore, human-centred research approaches should always be flexible, allowing appropriateness to be the key determining factor in the selection of...