Will People Talk to AI Survey Agents?

In the early 1960s, MIT professor Joseph Weizenbaum introduced the first artificial intelligence (AI) chatbot, modeled as a psychotherapist. The development sparked excitement as people saw machine learning algorithms capable of replicating human dialogue as the next frontier. Decades later, software engineers can train chatbots to emulate characters, engage in valuable dialogue, or gather feedback. The spread of generative conversational agents has transformed how people interact with technology and expanded the ways companies can solicit feedback from customers or conduct market research surveys. As more researchers develop AI-driven agents that interact with consumers, it is essential to examine whether people would genuinely engage with AI agents and what value these interactions bring.

How do people talk to AI survey agents?

Modern AI agents simulate natural conversations that approach human dialogue. By using adaptive techniques, the agents can make the survey experience more engaging and insightful than traditional static forms. AI agents dynamically tailor questions and follow-ups based on previous answers, demographic information, and participant interests. This approach enhances engagement, improves data quality, and often yields higher completion rates. Moreover, because AI agents can adjust questions to better understand and respond to participants, they yield critical insights that traditional techniques might have missed, as the process is more engaging and efficient.

Recently, researchers Max Lang and Sol Eskenazi tested an LLM-based telephone survey system at scale in the United States and Peru. Participants received web links to direct phone calls, where a virtual voice agent engaged in real-time, interactive, adaptive conversations that managed interruptions, provided encouraging messaging and ensured contextually appropriate dialogue. The agents conducted thousands of interviews, tailoring their approach to each participant’s unique responses and ensuring insightful outcomes. Impressively, these interviews were completed in about half the time while still yielding rich qualitative insights (Lang and Eskenazi, 2025).

This evidence suggests that people are willing to talk to AI agents. The scalability and efficiency of AI survey agents allow for high-quality research without the need for extensive interviewer recruitment or training. AI agents may provide better insights by optimizing for valuable perspectives, attitudes, and suggestions.

Do AI agents produce valuable responses?

AI interviewers use several response quality metrics to evaluate participant responses:

• Response Quality Flags: Machine learning models scan responses for low-quality response indicators, such as gibberish, profanity, short answers, and rushed answers. These responses can be flagged as poor quality, excluded from analysis, and may result in the participant not receiving a reward.

• Internal Consistency and Contradiction Detection: AI agents check for logical consistency across responses. Where they find the participant has contradicted themselves, the agent can dynamically produce follow-up questions to clarify based on previous answers. This often yields highly nuanced and deep responses that are more insightful to researchers.

• Sentiment and engagement analysis: Models analyze the sentiment of responses. Positive sentiment and high engagement can prompt more profound questions, while neutral or negative sentiment may lead to engagement-focused follow-ups tailored to the participant’s background, eliciting more valuable responses.

• Semantic coherence and lexical richness: AI agents assess responses for logical flow, consistency, and depth of vocabulary. These metrics tend to be consistent estimators for genuine responses and actionable suggestions.

  
  

Organizations that develop AI agents, such as Terac, use these metrics to ensure that survey responses are of high quality and provide valuable feedback. Indeed, developers can use interview quality to further train and refine AI agents to use improved techniques and create better incentives for good-faith participants. Because these metrics provide a positive feedback loop, researchers gain a deeper understanding of reviewing conversations based on tone, clarity, relevance, and quality, enabling the continuous refinement of the voice agent’s performance and ensuring that high-value, actionable, and worthwhile responses contribute to the final analysis.