User experience (UX) research is an essential part of creating products that are intuitive, user-friendly, and meet the needs of target users. To this end, UX researchers strive to understand user behaviors and needs by conducting research through various methods such as surveys, user testing, and data analysis. UX designers, on the other hand, incorporate research findings into the design processes to create products that satisfy the target users’ needs.
However, the release of ChatGPT has impacted the UX research and design landscape by automating and boosting productivity in UX-related tasks. As a result, this has sparked the debate of whether UX researchers and designers will be replaced by AI tools such as ChatGPT.
This review will be divided into two parts: UX research and UX design. In this article, we will discuss how ChatGPT revolutionizes the UX research landscape, where it falls short, as well as the implications of AI tools in the field of UX.
ChatGPT for UX research
To better understand how ChatGPT helps UX researchers at work, let’s assume the following scenario:
You are a UX professional in a tech company trying to create a new hotel booking application in Malaysia. Upon learning what ChatGPT can do, you are excited to incorporate ChatGPT in your workflow to see how it boosts productivity.
Ideation
1. Competitor Analysis/Benchmarking
Competitor analysis involves studying the user experience of competing products and is a crucial component of UX research. Both researchers and designers benefit from this process by studying the task flows, features, or app architectures of other similar apps, obtaining valuable insights pertaining to product strengths and shortfalls. This helps identify UX trends, design gaps and opportunities, as well as inform better design decisions.
As an AI language model, ChatGPT is capable of boosting this lengthy process by generating a summary of the competitive landscape within seconds. The model does this by analyzing its training data to identify common user feedbacks, pain points, and areas for improvement. All that is needed is a prompt to conduct the analysis, and ChatGPT can do the rest.
Prompt
“I want to create a hotel-booking app in Malaysia. Conduct a UX competitive analysis using Agoda’s app.”
2. Suggest User Profiles for Research
Selecting the right users for UX research ensures the insights obtained represent the target audience of a product. This enables the development of user-centered designs and products that meet target users’ needs and expectations.
ChatGPT can be helpful in this process to identify the potential user groups of a product by analyzing its training data about user preferences, behaviors, and demographics. Additionally, ChatGPT has the capability to offer recommendations on how to break down user groups when specific requirements need to be met and multiple user groups are involved. These help UX researchers to better target and understand the selected user groups of a new product, enabling them to conduct more effective research.
Prompt
“Imagine yourself as a UX specialist. I want to conduct a user research on a new hotel booking app in Malaysia. What kind of users should I recruit?”
3. Generate Product Use Cases
Understanding the target users’ behaviors and motivations is crucial to design user-centered products. Identifying the user personas and journeys reveal valuable information about users’ usual concerns, needs, and motivations when using a particular product. This also fosters a shared understanding of the user base and enables more effective communication among researchers, designers, and cross-functional teams.
ChatGPT’s extensive database empowers it to generate realistic user scenarios and provide insights into how users may interact with a product or its features. For instance, UX researchers can leverage ChatGPT to create user personas and analyze user journeys and pain points in competing products. This can be particularly valuable in the research preparation phase, as it helps identify research directions and generate ideas.
Prompt
“Create 2 personas of people who use hotel-booking apps in Malaysia.”
“What would be the key information Sarah (Persona 1) needs in each stage of the user journey? List down in point form.”
4. Suggest Research Methodologies
Due to the myriad of available methods that address different problems, choosing the right research approach can be daunting even for experienced UX researchers. Selecting the wrong research method can lead to wasted resources and expenses. What’s worse – using the wrong approach may yield biased findings, leading to flawed design decisions that negatively impact user experience.
This is where ChatGPT comes in handy – it is capable of identifying the ideal research methodology based on the objectives defined. This provides valuable guidance for UX researchers in choosing the most appropriate approach. For instance, you can define the objectives of your research and request ChatGPT to suggest an ideal research methodology that answers the objective.
Prompt
“My research objective will be to understand the users’ current experiences using other hotel booking apps, and also generate ideas on what kind of information is needed in my new app. What kind of user research method should I go for? List only the best one.”
Limitations
While ChatGPT can provide great assistance for UX research, its data may be unreliable due to biased and unvalidated information within its database. Additionally, the suggested research method and user group may not be the only solution to the research objectives.
Therefore, it is important for UX researchers to fact-check ChatGPT-generated information and not solely rely on them during the research planning phase. Project or product requirements must also be considered when selecting target user groups and research methods. While ChatGPT can be used as a reference, talking to real users remains a necessary part of the research to obtain relevant feedback and make better design decisions.
Research
1. Generate Interview Questions
ChatGPT’s capabilities are not limited to assisting with research planning and it is surprisingly useful during research execution too. ChatGPT helps brainstorm and generate interview questions for the researchers. This can be done by providing ChatGPT with some prompts along with the objectives for the interview questions, easy as that!
Back to our scenario of creating a hotel-booking app in Malaysia. During the preliminary rounds of research, you might be interested in users’ current experiences, pain points, and information needs while booking a room using hotel-booking apps. ChatGPT can assist by suggesting potential questions and even help with structuring the question flow.
Prompt
“I would like to draft some questions for a user interview session. The questions should start with generic questions like their current experiences using hotel-booking apps, then narrow down to specific questions about their pain points and content requirements. Draft in total 10 questions.”
2. Translate Test Scripts
As UX researchers conduct studies in countries that speak different languages, translation becomes an inevitable aspect of research. Properly translated test scripts (such as interview questions) are essential to help local users understand the questions asked and provide better feedback.
Translation tasks can be daunting as the traditional translation process can take hours, if not days to complete. However, with ChatGPT’s instantaneous translation capabilities, the process can be completed in seconds. This not only saves time and resources but also ensures that research questions are accurately translated, thus enabling researchers to obtain accurate and meaningful data.
Prompts
“Help me translate the questions above to Chinese.”
3. Suggest Information Architecture
Information architecture helps organize and structure information on a website or app in a logical manner that enables users to easily find the information they need. This reduces the chance of users becoming frustrated and abandoning their tasks, resulting in bad user experience.
When designing a new app or incorporating new information into existing websites or apps, ChatGPT has the capability to suggest the optimal structure of how information should be arranged. This is made possible by its vast database of patterns and information structures collected from numerous sources. By utilizing ChatGPT’s suggestions, UX researchers can further validate these structures with methods like card sorting or tree tests to ensure the best possible user experience.
Prompt
“I want to create a hotel-booking app for local Malaysians. Assume yourself as one and tell me what is the ideal information architecture for the following information: (About Us, promotions, search bar for hotel booking, customer reviews, profile settings, terms and conditions, chat support, information for specific hotels, hotel pricing)
Arrange them into at least 2 levels of information, where Level 1 are information users must see upfront, and level 2 refers to information users will see after clicking into certain sections or buttons.”
Limitations
ChatGPT suggests interview questions and information architecture structures based on common patterns observed from its database rather than analyzing the “best” solution for each inquiry. Therefore, UX researchers should only use them as a reference and not rely on them completely.
Besides, the model may not capture the cultural nuance and understanding that a human translator can bring to the table. As a result, the translations produced by ChatGPT may not have the necessary cultural sensitivity required for effective communication in some situations.
Reporting
Summarize Data and Create Stories
Research findings become nothing more than a bunch of meaningless data if they are not presented in ways that resonate with the target audience. Storytelling is a necessary skill to UX researchers as it allows the audience to empathize with the users, thereby communicating research findings to team members and stakeholders effectively.
ChatGPT is capable of summarizing lengthy texts into easily comprehensible points and creating user stories that cater to various audiences based on the tone and context set. With this, UX researchers can now seamlessly craft user stories that strike a balance between quantitative and qualitative aspects.
For example, you would like to share your findings with cross-functional teams and stakeholders after a research project. To make it easier for the audience to understand, you can summarize the details into a short paragraph instead of a lengthy report by requesting a quick summary from ChatGPT using simple prompts as shown below.
Prompt
“Summarize the findings above to 1 paragraph.”
Once you have extracted the key information, you can also prompt ChatGPT to create stories that suit different audiences based on different requirements.
Prompt
“Create a story out of this data and tell it to a group of audience who are insensitive to numbers.”
“Now tell the same story to a group of stakeholders. Numerical data is fine.”
Limitations
ChatGPT is capable of condensing lengthy texts into summarized points and creating stories out of basic data. However, unlike human researchers, it falls short in considering contextual factors of the data, which are important for precise analyses.
The model focuses on the literal meaning of the content and does not have the ability to accurately capture contextual cues and information (such as users’ prior experiences and emotions). This may result in inaccurate conclusions of users’ feedback.
The Big Picture: UX Researchers Remain Unrivaled
Without a doubt, ChatGPT is a remarkable productivity tool even in the field of User Experience where human interaction is paramount. UX researchers can leverage it to boost productivity, streamline workflows and generate useful research insights for designing user-centric products. Various ChatGPT prompts for user research can help support the process.
However, while ChatGPT serves as a promising tool for UX researchers, here are some limitations of the model that we must not overlook.
1. Lacks understanding of human emotions and contexts
One major limitation of ChatGPT is its lack of sensitivity to human emotions and contexts. While the model is claimed to be well-versed in analyzing user sentiments from text-based input, it lacks accuracy when it comes to ambiguous situations such as when users provide mixed or subtle feedback. This is because the model was not trained to understand different contextual and cultural knowledge. As a result, it can only offer generic responses instead of context-specific answers, which can pose a challenge in the field of UX where customized experiences are vital to enhance user experience.
2. Uncontrolled sources and biased suggestions
Besides, another limitation of ChatGPT lies in its limited knowledge and potential to generate biased responses. The model’s data only goes up to year 2021, which prevents it from providing responses that incorporate the latest developments and changes in specific fields. Besides, it is also incapable of deliberately filtering biased data, resulting in the possibility of generating biased responses.
Our Verdict?
ChatGPT as it is currently, has noteworthy flaws and shortcomings which stand in stark contrast to the contributions UX researchers can bring to table.
In our next article, we will explore the role of ChatGPT in UX design, and the challenges UX designers may face with the rise of AI tools.
