Record how it was made

Tags: provenance, metadata, audit

Generated things get edited, copied, pasted into documents, attached to email and merged into decks. The conversation that produced them does not come along. What arrives six months later is an artifact with no account of itself, in a form indistinguishable from something a person wrote and checked.

The artifact outlives its context, always

This is not an edge case to design for. It is the normal lifecycle of every useful output a product generates.

The draft gets pasted into the doc. Someone edits two sentences. The doc gets attached to an email. A figure from it goes into a deck. The deck gets forwarded. At no point does anybody act in bad faith, and by the end the number on slide 9 has no recoverable relationship to the model, the inputs, or the person who changed it.

Then someone asks where it came from, and the honest answer is that the information existed in a chat history that nobody can find and that may not have been retained.

Fields, not prose

The fix is a structured record attached to the artifact. Structured is load-bearing: a record you cannot filter, compare or verify is a story about the artifact rather than a property of it.

Five fields cover most of it:

Model and version. "GPT-4o" is not a version. The thing that produced this output no longer exists under that name alone.
Inputs. Which documents, records or sources it drew on. Identifiers, not descriptions.
When. Because the same model and the same inputs produced something different last Tuesday.
The request. The prompt or a hash of it — see the fork below, because this is the field that can be the problem.
The human-edit delta. Which spans are no longer the model's output.

The field everyone omits is the valuable one

That last one. Almost nothing records it, and it is the one that answers the question people actually have.

It is the difference between a sentence somebody stands behind and a sentence nobody has read. A document that is 90% model output with ten edited sentences is a very different object from one a person went through line by line, and from the outside they are identical. Recording the delta makes the distinction recoverable — which is exactly what mark what the product vouches for needs in order to work on an artifact that has left the product.

It also has a pleasant second property: the edits are evidence of review. A span a human touched has been read. A span nobody touched may have been read or may not. That is weaker than an explicit sign-off and far better than nothing, and it costs you nothing to capture because the editor already knows.

It has to survive the export

A provenance record that lives in the product's database and not on the artifact solves the easy half of the problem. The artifact is the thing that travels, so the record has to travel with it — in file metadata, in a sidecar, in the document's own structure, in the copied payload.

If the record is lost the moment someone copies to clipboard, it protects the case where nothing went wrong and fails the case where something did.

When not to

When the artifact is genuinely ephemeral. A chat reply nobody keeps needs no dossier. The test is whether it can be copied out; if it can, assume it will be.

When the provenance is more sensitive than the artifact. This is the real fork and it is not obvious. Recording the prompt means recording whatever was in the prompt — which may be confidential input, personal data, or a question whose existence is itself the secret. A provenance record can be a disclosure surface. In those cases store a hash rather than the text, or record the class of input rather than the input, and treat the record as carrying the sensitivity of the thing it describes. See answer within the boundary: the same caution, one level down.

Grounded in

EarthWonders and RockStore model specimen provenance as a structured record rather than a free-text note. Both are marketplaces for fine minerals, and in that hobby where a piece came from drives much of its value — locality, the collection it passed through, when it was recovered.

Free text would have been the obvious and much cheaper choice, and it would have been decorative. You cannot filter on it, cannot compare two specimens by it, and cannot verify any part of it. Modelling it as fields gave a listing the standing of a catalogue entry rather than a description, and made locality a browsable dimension of the whole marketplace instead of a paragraph buried in each item.

Generated artifacts have exactly the same property — their origin determines how much weight they can carry — and they almost always get the free-text treatment, when they get anything at all.

Anti-patterns

"AI-assisted" as the whole record. A binary where a record was needed. It tells you a model was involved and nothing about what, from where, or changed by whom.
Provenance in the chat log. Technically retained, practically unfindable, and gone the moment the artifact leaves.
Free text instead of fields. Unfilterable, uncomparable, unverifiable.
A record that dies on export. Protects the uninteresting case and fails the interesting one.
Recording the model but not the human edits. Loses the one distinction anyone downstream actually needs.
A watermark as a substitute. A watermark answers "was this generated" — a useful but different question. It cannot answer "from what", "by which version", or "which parts has somebody checked", and treating the binary as sufficient is how a product ends up with disclosure and no provenance.

The smallest version worth building

Five fields on the artifact, written at generation time, copied on export. Model and version, input identifiers, timestamp, a hash of the request, and a list of spans a human has since edited.

The last one is the only part that needs new work, and it is the one worth doing first. Everything else you already know at the moment you generate; you are simply declining to write it down.

Mark what the product vouches for — what this record is for: the edit delta is how vouching survives the artifact leaving the product.
Keep the evidence on screen — sources while the reader is in the product; this is the same sources after it has left.
Regeneration is an edit, not a retry — if a second attempt is a new artifact, it needs its own record rather than inheriting the first one's.