If you run user research, you've likely asked yourself: Can Synthetic Testers really think like my customers?
We decided to answer that question with a head-to-head study.
We gave 25 real shoppers and 25 Synthetic Testers the same assignment: buy a cordless vacuum on Amazon for under $300, choosing between Dyson and Shark. Same website, same budget, same instructions.
The headline: Synthetic Testers chose the same brand as their matched human shopper 84% of the time, and both groups selected the same winning brand.
Just as important is what Synthetic Testers did beyond that. They researched more thoroughly and surfaced an insight the human shoppers missed. Here's the full story.
The challenge: $300, two brands, one decision
We chose a purchase that people genuinely think through. A cordless vacuum is a meaningful expense, the specs are hard to compare, and most shoppers already have an opinion about Dyson and Shark.
Our 25 human shoppers were US adults aged 25 and older who use a vacuum at home and help make household buying decisions. There were no survey questions and no simulated store. Every session took place live on Amazon.com, with participants thinking out loud as they shopped.
The brief was simple:
You have a $300 budget for a cordless vacuum. Compare Dyson and Shark on Amazon, choose one, and tell us why.
Each shopper completed the same steps:
- Find a Dyson cordless vacuum for $300 or less and add it to the cart.
- Find a Shark cordless vacuum for $300 or less and add it to the cart.
- Compare the two and keep only the winner in the cart.
- Explain the decision. No order was placed.
Then 25 Synthetic Testers completed the identical task, on the same website, guided by the same AI moderator.
No made-up personas: built from real shoppers
Many synthetic research tools start with a hand-written "typical customer" persona. That approach risks testing your own assumptions instead of your customers' behavior.
We took a different route. We ran the human study first. TheySaid's AI Research Assistant then reviewed every session: what people said, what they clicked, which products they compared, and what ultimately drove their decision.
From that behavior, it identified the distinct shopper mindsets in the group. None were defined in advance. Each mindset became a set of inputs for a Synthetic Tester:
- Motivation: what the shopper is trying to accomplish and what matters most.
- Worldview: the attitudes, biases, and trust that shape how they judge products and brands.
- Knowledge: what they already know about the category before they begin.
Each Synthetic Tester was then matched to one human shopper, so we could compare results person by person and study by study.
The foundation is video. Text captures what someone said. A screenshot shows what they saw. Video shows what they saw, what they did, what they said, and why they decided.
The results: 84% chose the same brand
The final brand split came down to just two shoppers, and average confidence in the final choice was almost identical across both groups.
Both studies also arrived at the same major themes independently:
- Price leads. Value was the number one purchase factor for humans and Synthetic Testers alike.
- Reviews build confidence. Ratings and review volume served as social proof.
- Pet hair is a deciding factor. Anti-tangle claims received close scrutiny.
- Brand trust matters. Reputation shaped how both groups judged their options.
- Negative reviews change decisions. A single criticism could shift a shopper's choice in either group.
Humans stopped at "good enough." Synthetic Testers kept digging.
Both groups cared about the same things. The difference was in how thoroughly they researched.
Human shoppers were quicker and more selective. Many relied on the price and star rating and stopped once an option seemed good enough. They also brought personal habits, past brand experience and design preferences into the decision.
Synthetic Testers were consistently thorough. They took more time before deciding, read beyond the star rating, questioned vague marketing claims and compared more options before committing. For a research team, that consistency matters: every session produces a complete, evidence-based account of the decision.
Why didn't every pair choose the same brand? Part of the answer is that Amazon personalizes the shopping experience. Two shoppers using the same decision logic can see different listings and prices, and reach different outcomes.
The insight human shoppers missed
Because Synthetic Testers don't skim, they surfaced details most human shoppers passed over:
- They compared more products. Synthetic Testers often evaluated 2 to 4 vacuums per brand. Most human shoppers looked at 1 or 2.
- They read beyond the stars. They went into review summaries and rating breakdowns.
- They asked for proof. They repeatedly looked for battery runtime by power mode, actual suction numbers and clear warranty length.
That last finding is valuable for any product team. It signals that shoppers may want concrete specifications rather than broad claims like "powerful suction." It's exactly the kind of hypothesis a team can take forward and validate with customers.
Both studies led to the same business recommendations
Research is only as useful as the decisions it supports. Hand either study to a vacuum brand, and the team would come away with the same four priorities:
- Lead with pet-hair and anti-tangle benefits.
- Make value comparisons easy.
- Strengthen review proof.
- Fix confusing product listings.
The human study added emphasis on Prime pricing, design and brand loyalty. The synthetic study added emphasis on specification details and review evidence. The direction was the same.
So, can Synthetic Testers shop like real people?
When they're built from real human behavior, the answer is: remarkably closely.
Synthetic Testers matched the big picture: the same winning brand, the same priorities, most of the same findings and the same business recommendations. On top of that, their thoroughness surfaced a new insight the human study didn't. Human shoppers contributed what only people can: emotion, personal taste and lived experience.
That's why the strongest approach isn't human or AI. It's human and AI. Synthetic Testers let teams go deeper, test more ideas and pressure-test their thinking. Real participants add the human nuance.
No data will convince you like trying it yourself. Run real users and Synthetic Testers side by side on your next project, and compare the results.
Try Synthetic Testers for free · Download the full research report
FAQs
What are Synthetic Testers?
Synthetic Testers are AI participants that complete research tasks the way a real user would: browsing a live website, thinking out loud and explaining their choices.
How were the Synthetic Testers in this study created?
From real people. TheySaid's AI Research Assistant analyzed 25 recorded human shopping sessions and turned the patterns it found into Synthetic Testers, each matched to one human shopper.
How closely did they match real shoppers?
84% chose the same brand as their human match. They reproduced 76% of their human counterpart's top purchase factors, and 70% of the human study's major findings appeared in the synthetic study.
Do Synthetic Testers replace human research?
They work best alongside it. Synthetic Testers captured the big picture and added deeper research, while human participants contributed emotion and real-world nuance.
