What the user is allowed to trust
Every AI interface makes a claim about how reliable it is, and most make it by accident. The model produces one stream of text; the interface renders all of it in the same font, in the same box, in the same confident voice — so the user is invited to believe the quoted fact and the invented one to exactly the same degree. Deciding which parts of an answer the interface will vouch for is a design decision, and if nobody makes it deliberately it gets made by the default styles.
The output is not one kind of thing
A single paragraph of model output routinely contains several different epistemic objects. Some of it is copied from a document you supplied. Some of it is a summary of that document, which may be a fair reading or may not. Some of it is a general claim from training. Some of it is a plausible sentence produced because the shape of the answer needed one there.
Those have genuinely different reliability, and the user has no way to tell them apart, because nothing in the presentation distinguishes them. The interface has flattened four kinds of claim into one voice.
That flattening is the actual problem. It isn't that the model is sometimes wrong — that's a given, and it's a different design problem — it's that the product offers no way to treat the shaky parts differently from the solid ones.
Three things an interface can do with a claim
| What it means | Right when | |
|---|---|---|
| Assert | Present it plainly, as the product's own statement | You'd be willing to be held to it — it's deterministic, or checked, or the cost of being wrong is trivial |
| Attribute | Present it with its source attached and reachable | The claim came from somewhere specific and the user can check in one click |
| Withhold | Don't present it as an answer at all | You have no basis for it, and a plausible sentence would be worse than a gap |
Most AI features use the first for everything. The interesting work is deciding, per surface, which claims move to the second row — and being willing to use the third at all, since an empty state that says the product doesn't know is a harder thing to ship than a paragraph that sounds fine.
Confidence scores are the wrong instrument
The usual answer to this is a number: 87% confident. Two problems.
The first is calibration. A model's reported confidence is frequently not a probability of being right in any usable sense, so the number is often decoration with a decimal point.
The second is that people can't act on it even when it's honest. Told a claim is 73% likely, a user has no procedure. Nothing in that number tells them what to do next, so in practice they either ignore it or treat anything above a vague threshold as true.
What people can act on is different: where it came from, and what happens if it's wrong. A citation that opens the source passage gives someone a concrete next move. Knowing an action is reversible tells them how much to care. Those are things a person can do something with; a percentage is not.
The ladder from sensitive data
There's an existing pattern worth stealing here. For sensitive values — medical records, account numbers — the discipline is well established: show the minimum necessary by default, make the full value a deliberate act to reveal, and for the most consequential cases record that the reveal happened.
The same three moves apply to model claims:
Minimum necessary. Show the least the user needs to act. Not the whole retrieved chunk, and not the model's reasoning by default. Excess detail reads as thoroughness and functions as noise.
Deliberate reveal. The evidence should be one intentional click away, and it should be the real thing — the actual source passage, not a summary of it. A citation that expands into another paragraph of generated text is worse than no citation, because it looks like verification while adding another layer of the same uncertainty.
Record what was claimed. For anything consequential, keep what the system asserted, what it was based on, and when. Partly so a wrong answer can be traced, and partly because a team that can't reconstruct what the product said last Tuesday cannot improve it.
Trust runs in both directions
The other half of this is what the user is asked to trust the product with, and that constraint can be more binding than accuracy.
On DocRobot, an enterprise workspace for confidential documents, the blocker was never quality. It was that legal would not permit the documents near a hosted model at all. So redaction became the first step in the flow rather than a setting — sensitive content removed locally before anything was sent. The trust model was the product rather than a feature of it.
That case is unusual only in how early the constraint appeared. Any product handling someone's real material has a version of it, and the answer is generally the same: decide what leaves the machine before designing what comes back.
The failure mode of all this
Applied without judgement, this produces a product that hedges everything — confidence badges on every sentence, warnings stacked on warnings, a disclaimer under each answer. That isn't trustworthy, it's exhausting, and it hands the judgement back to the user, which is the work they came to the product to avoid.
The test I'd use: does this piece of trust design cost anything when the model is right? A citation sitting quietly next to a claim costs nothing and pays off when someone checks. A modal asking whether you're sure costs something on every single interaction, including the overwhelming majority where nothing was wrong.
Decide what the product will vouch for. Show the evidence for the rest. Say nothing where there's no basis for a claim — and let all three fade into the background when the answer is simply right.
Related: design the failure state first, and how I work.