A research-on-research involving 2,006 participants exploring the issues with traditional surveys and how a conversational research approach might address those problems.
Reach3 Insights ran a parallel study using the same topic and questions but with two different methods.
1,001 participants in total
Received a traditional, email-based online survey
1,005 participants in total
Received a chat-link, mobile-first experience using Rival’s conversational platform
“We wanted to validate how far conversational methods have come, and what that evolution means for researchers today. Inside this report, you’ll find practical insights and best practices that anyone can use to improve how they engage with participants and elevate the quality of the insights they gather.”
Jennifer Reid, Co-CEO and Chief Methodologist, Rival Group
When people respond via a conversational survey, they say more.
And the quality is better. Using our proprietary Thoughtfulness Score (a metric that evaluates depth, relevance, clarity, emotional tone and overall value of a response), we quantified the quality of the open-ends from both groups.
Single- and multi-select questions were remarkably consistent across both methods. The data points showed small, statistically insignificant differences, suggesting that a conversational format does not distort core measures. It preserves data integrity while providing a better respondent experience (and richer outputs).
“I like it and it’s not like I’m just filling out a quiz in the back of a school classroom. It’s like I’m actually having a conversation.”
– US, 18–24, She/her
“It felt like just a regular text conversation. That made me feel more invested in it and I felt comfortable, like I was texting a friend.”
– Canada, 25-34, She/her, Facebook
“It feels more like a conversation. Questions pop up more quickly than having to wait for the page to change, and the format of the questions does not change.”
– US, 45–54, He/him