Redaction moves to the export

Tags: privacy, on-device, enterprise

A product organised around "nothing leaves the machine unredacted" looks like it loses its reason to exist the moment a capable model runs locally. That conclusion is wrong, and it is the part worth writing down.

The risk was never that a model saw the content. It was that the content crossed a boundary. Local inference does not delete the boundary, it moves it.

Where the boundary goes

When inference happened somewhere else, the boundary was the network call, so the redaction work sat at the input. When inference happens on the user's machine, the boundary becomes the export: what gets copied out, pasted into a ticket, attached to an email, shared with a colleague who isn't cleared for it.

Same craft, different position in the flow.

When to reach for this

• A model moved on-device, and the redaction step in your flow now looks like dead weight.
• The product handles material someone else owns — client documents, patient records, anything under a review.
• The real constraint on adoption is legal or security, not accuracy or speed.

When it doesn't apply

• The content was never sensitive, in which case both boundaries are imaginary and the ceremony costs you.
• The user is the only person who will ever see the output, and there is no export path at all — rare, and worth checking rather than assuming.

Grounded in

DocRobot was an enterprise workspace for teams whose documents are confidential enough that legal will not let them near a hosted model. It started from a rule rather than a feature: nothing leaves the machine unredacted. Every other decision followed from that one.

In practice that meant redaction was the first step in the flow rather than a setting somewhere in preferences. In the working proof of concept it was manual search-and-redact — slow, unglamorous, and the reason a security review could pass at all. Later iterations detected names, dates and account numbers automatically. The document stayed on screen throughout and answers were grounded in it, so a team could trace any response back to the page it came from.

The whole shape of the product was a workaround for the model being somewhere else. That constraint has since dissolved — and the craft survived it by moving.

Anti-patterns

• Deleting the redaction step when the model goes local. The most tempting move, and the one that quietly removes the control at the exact moment the content becomes easiest to move around.
• Redacting at the output of the model instead of the input, while still using a hosted one. By then the content has already crossed.
• Treating the trust model as a preference. A checkbox in settings is not a security review, and it will not pass one.

Related patterns

• Design the failure state first — "saw what it shouldn't" is the one failure you cannot design a recovery for.
• Mark what the product vouches for — the same question pointed the other way: what the user trusts the product with.

The full argument: Designing document AI before the model could run locally.