Synthetic Users
DISCOURSE REVIEWS IN UX RESEARCH
SyntheticUsers went ahead and stood up to say it, "User research without the users."
People have thoughts!
Here’s one: "User research without the user isn't user research; it's desk research" (Rosala & Moran, 2024). I wouldn’t kick desk research out of bed for eating crackers, but that’s a fun burn.
Kate Moran, VP of Research & Content at Nielsen Norman Group
The use of synthetic users in UX research has sparked a lively and sometimes spicy debate among industry professionals, with opinions ranging from enthusiastic support to cautious skepticism. This article explores the diverse perspectives on synthetic users, their potential benefits, and the areas of contention that have emerged within the UX community.
Disclaimer: You will notice below the primary sources (e.g., NN/g) are identified as the owners of their rhetorical positions rather than the authors of the sources alone. Although authors are named in references, the conversation we're examining is hosted and nurtured in curated spaces. My scope is there for this exercise.
What are synthetic users?
Synthetic users are AI-driven entities or artificial constructs designed to simulate user interactions in research. I like to think of them as interactive personas. They are employed to replace or supplement human participants, providing scalable and efficient solutions for testing and analysis. Synthetic users utilize synthetic data and are AI-generated to replicate user actions and preferences, thus enhancing AI capabilities and enabling large-scale testing (SyntheticUsers.com, 2023).
However, while synthetic users offer many advantages, they also present significant limitations. Their lack of real human insight can reduce research authenticity, as they fall short in capturing the nuanced emotional and contextual aspects of real user experiences (Rosala & Moran, 2024; Reddit Users, 2023). Therefore, while they are valuable tools, they cannot fully replace the depth and authenticity that human participants bring to user research. This makes it essential to consider the balance between synthetic and real user interactions to maintain the integrity and depth of research outcomes (Wiedmaier, 2023; Arora, 2023).
The spread
Pro-Synthetic Users
SyntheticUsers.com
Sentiment: Strongly Pro-Synthetic
Opinion: Emphasizes the efficiency, scalability, and privacy benefits of synthetic users. The platform promotes the quote "User research without the users," highlighting their approach to integrating synthetic users into UX research (SyntheticUsers.com, 2023).
ACM Interactions
Sentiment: Pro-Synthetic
Opinion: Argues that synthetic users can increase data diversity and reduce biases, crucial for fields with data scarcity (Li, 2023).
Towards Data Science
Sentiment: Pro-Synthetic
Opinion: Highlights synthetic data's role in enhancing AI capabilities (Koc, 2023).
Neutral with Caution
HeyMarvin
Sentiment: Pro-Synthetic with Caution
Opinion: Supports the use of AI tools for automating data collection and analysis while cautioning against over-reliance on AI, stressing the need for human interpretation (Arora, 2023).
NNG Group
Sentiment: Cautious, leaning towards Anti-Synthetic
Opinion: Emphasizes the potential benefits of synthetic users while cautioning that "User research without the user isn't user research; it's desk research," stressing the importance of real human interactions (Rosala & Moran, 2024).
Anti-Synthetic Users
r/UXResearch
Sentiment: Anti-Synthetic
Opinion: Expresses concerns that synthetic users cannot capture the emotional depth and contextual understanding of real human participants, crucial for comprehensive UX research (Reddit Users, 2023).
UserInterviews.com
Sentiment: Anti-Synthetic
Opinion: Points out inconsistencies and quality issues in many AI tools marketed for UX research, suggesting these tools may hinder the research process (Wiedmaier, 2023).
Areas of agreement
Both proponents and some critics acknowledge the efficiency and scalability benefits of synthetic users, especially in automating repetitive tasks and enabling rapid prototyping and testing (Arora, 2023; SyntheticUsers.com, 2023; Rosala & Moran, 2024).
There is consensus on the privacy benefits of synthetic users, as they eliminate the need for real user data and reduce privacy risks, particularly valuable in sensitive fields (SyntheticUsers.com, 2023; Rosala & Moran, 2024).
Several sources agree that synthetic users can enhance data diversity and reduce biases, addressing gaps in real-world data collection (Rosala & Moran, 2024).
Area of greatest disagreement
Authenticity and depth of insights
Pro-Synthetic Users
SyntheticUsers.com, HeyMarvin, and TDS argue that synthetic users can provide valuable insights and help scale research processes. They emphasize the ability of synthetic users to mimic real user behavior and generate diverse data sets, which can enhance the overall research quality (SyntheticUsers.com, 2023; Arora, 2023; Koc, 2023).
Anti-Synthetic Users
NNG Group, UserInterviews.com, and r/UXResearch discussions express strong reservations about the authenticity and depth of insights generated by synthetic users. They argue that synthetic users lack the emotional depth and contextual understanding that come from real human interactions. (Rosala & Moran, 2024; Wiedmaier, 2023; Reddit Users, 2023).
Opportunities for Cooperation and Learning
Combining Strengths of AI and Human Researchers
Developing hybrid research models that utilize synthetic users for initial data collection and analysis, followed by validation and deeper exploration by human researchers, can ensure both efficiency and depth. This approach leverages the scalability of AI while retaining the nuanced understanding humans provide (Arora, 2023; SyntheticUsers.com, 2023; Rosala & Moran, 2024).
Enhancing Empathy and Contextual Understanding
Incorporating contextual data and real-world scenarios into the training of synthetic users can help bridge the gap between synthetic and natural user interactions. Developing tools that facilitate seamless collaboration between AI and human researchers can enhance the research process (Li, 2024; Koc, 2023; Rosala & Moran, 2024).
Cross-Disciplinary Learning
Encouraging cross-disciplinary learning and collaboration through workshops and conferences can promote a more holistic understanding and application of synthetic users. Sharing detailed case studies and experiences can help identify best practices and common pitfalls, leading to improved methodologies and outcomes (Wiedmaier, 2023; Koc, 2023; Rosala & Moran, 2024).
Conclusion
By recognizing the strengths and limitations of synthetic users, UX researchers can develop hybrid models that combine the efficiency of AI with the depth of human insight. Cooperation and continuous learning across disciplines will ensure that synthetic users are used effectively and ethically, ultimately enhancing the quality and impact of UX research.
Exploring this conversation, I was inspired that, between the marketing-driven reductive value propositions and the defensive-feeling call to arms, there is a community of inventive, well-intentioned humans trying to help.
I’m psyched about what synthetic users can become and happy to approach their existence with optimistic hypotheses and a relentless willingness to learn better.
References
- Arora, K. (2023). *AI in UX research: The good, the bad and the final verdict*. HeyMarvin. Retrieved from https://heymarvin.com/resources/ai-in-ux-research/
- Rosala, M., & Moran, K. (2024). *Synthetic Users: If, When, and How to Use AI-Generated “Research”*. NNG Group. Retrieved from https://www.nngroup.com/articles/synthetic-users/
- Li, J. (2024). *How far can we go with synthetic user experience research?* ACM Interactions. Retrieved from https://interactions.acm.org/archive/view/may-june-2024/how-far-can-we-go-with-synthetic-user-experience-research
- SyntheticUsers.com. (2023). *User research without the users*. Retrieved from https://syntheticusers.com
- Wiedmaier, B. (2023). *The future of UX research is brighter with AI*. UserInterviews. Retrieved from https://www.userinterviews.com/blog/ai-panel-recap
- Koc, V. (2023). *Creating synthetic user research using persona prompting and autonomous agents*. Towards Data Science. Retrieved from https://towardsdatascience.com/creating-synthetic-user-research-using-persona-prompting-and-autonomous-agents-b521e0a80ab6
- Reddit Users. (2023). *Synthetic Users*. Retrieved from https://www.reddit.com/r/UXResearch/comments/12sfz39/synthetic_users/