Synthetic Buyers: Testing Products and Creatives at the Speed of Software
I gave a talk at SAMI (Sales and Marketing Intelligence) on synthetic consumers, a technique that uses LLMs to simulate buyer panels for market research. Below is a summary of the main points. Watch the full talk here:
The Problem
When I was at HelloFresh, we had an $800M+ marketing budget and my VP of Brand wanted to test a full rebrand. We couldn’t A/B test something that sweeping. So we commissioned a research firm, ran interviews, assembled panels. Months of work. A small fortune. And the resulting personas ended up in a PowerPoint that nobody opened again.
This is the norm. Consumer research is slow, expensive, limited to small sample sizes, and the output is a static artifact you can’t iterate on.
How Synthetic Consumers Work
You prime an LLM with a demographic profile (age, income, location, political leaning, past purchases) and ask it research questions as if it were that person. Scale this to thousands of different profiles and you get a virtual panel that approximates a real population.
The naive version is just prompting ChatGPT with “You are Pam, you live in North Brooklyn, you earn $85K.” In practice, there is more sophistication behind it. But the principle holds: you can survey a simulated population at the speed of an API call instead of waiting months for a research firm.
Unlike traditional ICPs (which are top-down, built from internal assumptions about who your customer should be), synthetic consumers work bottom-up. You present a product concept to a broad simulated population and see who actually responds. Then you find your customer segments in the data.
Validation: Does It Actually Work?
In a published study we did at PyMC Labs with Colgate-Palmolive, we tested 57 oral care products (real, unreleased, and AI-generated concepts) with 9,300 synthetic consumers:
- Rank correlation with human panels: 0.72. Two separate human panels asking the same questions would correlate at about 0.90. Synthetic consumers get to ~80% of that ceiling.
- Distribution of ratings matched well, not just the averages.
- Synthetic responses were ~2x longer than human ones. LLMs are chattier, but this is actually useful: you get more reasoning about why a product was liked or disliked.
We also stress-tested with absurd products (a “dental microbiome fortifying elixir”). Real humans rated it generously (scores skewed toward 4-5). The synthetic consumers called it “expensive and a bit gimmicky.” Human panels have a documented positivity bias in surveys. LLMs, with proper prompting, don’t.
What You Can Use It For
This is a pre-screening tool, not a replacement for A/B testing. You still need real experiments to measure actual lift. But for narrowing down what’s worth testing live:
- Product concept testing. Colgate used it to triage hundreds of AI-generated concepts and decide which ones to prioritize for production.
- Creative pre-screening. Direct mail, video ads, Super Bowl spots: anything expensive to produce is worth pre-testing virtually.
- USP stress testing. For a UK brand (Flo) entering the US market, we simulated consumers in a CVS store and removed selling propositions one by one to find which ones actually drove purchase preference.
- Price elasticity. “Would you buy this at $5.99, $6.99, or $9.99?” across thousands of simulated consumers segmented by income.
- Taglines and positioning. Generate hundreds of options with AI, screen them synthetically, run live tests only on the top performers.
Connecting to Media Mix Models
For organizations already measuring marketing with Media Mix Models, synthetic consumers close a loop: use them pre-launch to prioritize campaigns, use MMMs post-launch to measure actual performance, then feed those results back to calibrate the synthetic panel for the next round. We’re building this with some of our clients now.
Where to Start
Two paths depending on your needs:
SaaS platforms for self-serve testing: Delve AI and Synthetic Users (virtual surveys), Artificial Societies (complex scenario simulations), Deli (digital twins using your CRM data).
Consulting firms for tailored methodology: PyMC Labs, Empathy Labs, or Bain can validate the approach against your data and build custom pipelines. This matters for B2B, where your buyers are niche populations that generic demographic data won’t cover.
A Bain study on synthetic customers reports ~50% reduction in testing time, one-third the cost, and 85% agreement with human panels.
What’s Still Hard
Niche audiences. If your buyers are 1,500 heads of clinical operations at large pharma companies, there’s not enough public data for an LLM to simulate them reliably. You’d need traditional quant data first to calibrate.
Temporal drift. Enterprise attitudes toward AI shifted fast over the past year. A panel built on data from nine months ago could be off-target. One direction we’re exploring: giving synthetic consumers agent capabilities with web search so they can stay current.
Geography. Most data coverage is US-centric. I’d want to do more work before making accuracy claims about European or Asian markets.
No effect size measurement. You can tell which creative is directionally better. You can’t measure the precise percentage lift. For that, you still need real users.
If you want to try it, we built a free concept app at synthetic-consumers.pymc-labs.com where you can upload creatives and test them against virtual panels. And if you have questions about applying this to your business, get in touch.