A cold product can't be judged

Tags: demo data, cold start, review

Dashboards, maps, feeds and marketplaces are designed for volume and reviewed at zero. An empty product cannot show the problems that decide whether it works — so every decision made against the empty version is made against a product nobody will ever use.

What only shows up when it is full

The failures that matter in a data-shaped product are density failures, and density is exactly what an empty state hides:

Overlap. Markers on a map that sit on top of each other; cards that collide once there are forty of them.
The long tail. The one entry with a name three times longer than the rest, which breaks the layout for everybody.
Distribution. Real data is uneven — a few categories with most of the items, most categories with a few. A view designed on evenly spread samples looks balanced and is not.
Scanning. Whether anyone can find anything is unknowable until there is enough to search through.

None of those can be seen in an empty state, and none of them can be seen in three hand-typed rows either.

Hand-built fixtures flatter the design

The usual stand-in is a handful of example records typed by the team. They are the wrong shape in every way that matters: too few to reveal density, too regular to reveal the long tail, and — because someone chose them — too tidy. Every name fits. Every value is plausible. Every category is represented once.

That is not neutral. A design reviewed against flattering data will pass review, and then meet real data for the first time in production.

Generation made a realistic population cheap

Until recently, a realistic population was expensive enough that it rarely got built: somebody had to write hundreds of believable records by hand, or clean and anonymise real ones. A model can now produce a plausible population — uneven, awkward, full of edge cases — in minutes.

That changes what is reasonable to expect. A realistic population is now cheap enough to be treated as part of the design deliverable, not a nice-to-have for later.

Two rules keep it honest:

Generate for plausibility, not beauty. Ask for the uneven distribution, the unwieldy name, the record missing a field, the outlier. A generated dataset of perfect examples is a hand-built fixture with more rows.
Label it everywhere it could be mistaken for real. Synthetic data that drifts into a screenshot, a pitch deck or a metric is a fabrication with a head start.

Know what it can and can't test

A synthetic population is excellent for layout, density, flow, and performance — everything about whether the interface holds up with realistic volume.

It cannot tell you whether real people behave like the invented ones. A model's idea of a typical user is a stereotype, and a product tuned to it is tuned to a stereotype. Use generated data to make the product reviewable; use real sessions to find out whether it works.

Grounded in

Motive is a live fleet-tracking map, which is about as data-shaped as a product gets — and it had no real trucks behind it.

So it used an LLM to manufacture a plausible fleet. That data-generation step is what makes the map feel populated at all: the difference between an interface that can be looked at and judged — does it read at a glance, do the markers separate, does the motion make sense — and a handful of dots on an empty continent that tells a reviewer nothing.

Anti-patterns

Reviewing an empty state. Every density problem is invisible.
Three hand-typed examples. Too few, too regular, too flattering.
Generated data that is too clean. Perfect records reproduce the flattery at higher volume.
Synthetic data escaping unlabelled. Into screenshots, demos, decks and — worst — metrics.
Tuning to the invented users. Using a generated population to decide how real people behave.

The smallest version worth building

Generate a few hundred records for your main view, and ask specifically for the ugly cases: long names, missing fields, heavy skew, extreme values. Load them, label the environment as synthetic, and review the screen at that volume.

Most of what you find in the first ten minutes could not have been found any other way before launch.

Evaluation loops — a synthetic population makes the interface reviewable; a fixed set of real cases tells you whether a change helped.
Smooth the gaps, don't invent the points — synthetic data is fine when it is labelled as synthetic; the danger is invention that sits among real values.
Prototype the distribution, not the screen — running the interface across a realistic spread is the same instinct applied to review.