Design the failure state first
In a product built on a model, the interface is decided by what happens when the answer is wrong — not by what happens when it's right. So draw that screen first. If you design the happy path and then bolt on error handling, you get a product that is confident exactly when it shouldn't be.
This is the part of AI product design that's actually hard. The interface patterns everyone worries about — streaming text, citations, a stop button — are solved shapes, and a competent designer picks them up in a sprint. What doesn't transfer from deterministic software is that the same input can produce a different output tomorrow, so there is no single correct screen to design. There's a distribution, and your job is to make the whole distribution survivable.
Wrong is not one thing
"The model failed" collapses at least five different situations that need different interfaces. Treating them as one error state is why so many AI features have a single grey box that says something went wrong.
| Failure | What the user actually gets | The design that holds |
|---|---|---|
| Confidently wrong | A plausible, well-formed, incorrect answer | Make checking cheaper than trusting — citations that resolve, the source passage inline |
| Nothing to say | An empty or refused result | An empty state that says why and offers the next move, not a shrug |
| Slow | A long wait with no signal | Stream partial output; always offer cancel |
| Did more than asked | An unrequested change to their work | Undo, not a confirmation dialog |
| Saw what it shouldn't | Sensitive data sent to a model | Prevent at the input, never at the output |
The last row is the one people get wrong most often, and it's the one where "design the failure state first" stops being a slogan.
The failure you cannot undo
Most failure states are recoverable. A wrong answer can be corrected, a slow one waited out, an unasked change reversed. Data leaving the machine is not in that category — once a confidential document has been sent to a third-party model, no interface fixes it afterwards.
That constraint drove the whole shape of DocRobot, an enterprise AI workspace I designed for teams working with confidential documents. The blocker to adoption wasn't accuracy or speed. It was that legal and security would not permit the documents anywhere near a hosted model. The failure state — the model sees something it must not — had to be impossible, not merely handled.
So redaction became the first step in the flow rather than a setting somewhere. Sensitive content was removed locally, on the machine, before anything was sent; the trust model was the product rather than a checkbox in its preferences. In the first working version that redaction was manual search-and-replace, which is slow and unglamorous, and it was still the thing that let a security review pass. Later iterations detected names, dates, and account numbers automatically.
Designing that failure first produced a different product, not a safer version of the same one. Had we started from the happy path — upload, ask, get an answer — redaction would have arrived late as a settings toggle, and the security review would have killed it.
How you know it worked
Failure design is the part most likely to be quietly wrong, because nobody demos it. Two habits keep it honest.
First, write the failure cases down as examples before you build, with the actual bad output pasted in. A wrong answer you have written down is a design brief; a wrong answer you imagine is a reassurance.
Second, build an evaluation loop early enough to tell you whether a change helped. In deterministic software you can reason about whether a fix works. Here you can't — the behaviour is a distribution, and improving one case routinely degrades another. Without a way to measure, teams end up tuning prompts by anecdote and calling it iteration.
Where this goes wrong
Designing failure first has a real failure mode of its own: products that are so hedged they're useless. Confidence indicators on every sentence, warnings stacked on warnings, a confirmation before anything happens. That's not trustworthy, it's just tiring — and it pushes the judgment back onto the user, which is what they came to the product to avoid.
The test I'd apply: does the failure design cost anything when the model is right? Citations don't — they sit there quietly and pay off when someone checks. A modal asking "are you sure?" does, on every single interaction. Undo is nearly free; confirmation is a tax on the common case.
Design the failure state first. Then make sure it disappears when nothing has failed.
Related: what's actually different about designing an AI product, and how I work if you're building one.