Subscribe to On the dot – the only consumer insights newsletter you need to power up your conversations.
Five things to consider before investing in synthetic data


GWI
Before investing in synthetic data, you need to know what it’s based on, how it’s generated, and how well it can answer your questions. These five considerations will help you assess what a vendor is offering.
1. Can your vendor name what’s feeding the machine?
Synthetic data is only as good as what it starts from. Look for the three Rs: robust, representative, and recent. Robust means there’s enough of it to build from. Representative means it doesn’t crowd out the less mainstream voices. Recent means it has the context that’s needed, because old data misses what’s changed. If a vendor can’t show you all three, that tells you something.
Know what’s behind the results. Ask where the source data comes from, who it represents, and how recently it was collected. Your vendor should be able to explain all three.
2. How was the approach chosen, and how does it shape what you get?
Different methods have different strengths and limitations. What matters is knowing which one your vendor picked and why. You don’t need to be a data scientist to have that conversation, and a clear answer shows there’s a real choice behind the product.
Question the methodology. Ask which approach they’ve settled on and why, and how that shapes the outputs you receive.
3. How will your vendor show you how reliable the results are?
Every answer carries some uncertainty, and some are far more reliable than others. So ask what confidence looks like. Is it a score for every answer, or one for the whole survey? And why that choice? One overall number can hide the weaker results. A breakdown helps you see which findings you can rely on and which need further checking.
Ask what’s behind the score. Don’t settle for one overall score. Your vendor should explain what it measures, how it’s calculated, and where the results are less reliable. Push for more detail and greater transparency.
4. How well does it match your use case?
Start with what you need to find out. For mainstream audiences, synthetic data can help you:
- Get an initial steer or sense-check on an idea/concept
- Explore which messages might resonate with your audience
- Identify areas to explore further through fieldwork
Different question types tend to give different levels of confidence, so ask how your vendor has seen that play out and how they bring it together in the research you’re running. How questions are worded and who’s asked shapes how much you can rely on the results. The easiest way to see how it holds up for you is a pilot: take a study you’ve already run with real people, have the vendor answer the same questions, and compare.
Don’t skip over question design. Ask how they handle it, as it directly affects the confidence and quality of your results and check how well it works for your use case.
5. Can you see the simulated results at a respondent level, or only the total?
A total can look convincing and still hide what’s going on underneath. Respondent-level results let you look at the groups you care about, so smaller audiences don’t get lost in the average. That matters, because the best ideas often come from the niche. This one’s especially crucial. If a vendor can only show you the total, you can’t check that those audiences still look distinct.
Dive into distribution. Look beyond the overall result. Ask whether you can explore how answers vary across your audience, so differences between smaller groups aren’t lost.
Discover how GWI’s simulated data gives you answers you can trust.
Other resources


