A Model Spec Is Not a Runtime Attestation

Aug 19, 2026 7 min read

An agent that says “done” has made a claim about the world. It has not produced evidence.

That distinction sits underneath OpenAI’s August 18, 2026 Model Spec update. The release adds a section titled “Be clear about capabilities and limits” and clarifies how assistants should handle false or unsupported premises. Both are sensible changes. Both also expose the gap that agent infrastructure now has to close: a behavioral specification can tell a model not to overclaim, but it cannot attest that a file changed, a message was sent, a deployment succeeded, or a tool result was read correctly.

The model can describe its limits more honestly. The runtime still has to prove its work.

What changed on August 18

The official changelog lists four material changes: more detail on relational interactions with teens; guidance for false or unsupported premises; removal of outdated pre-reasoning-model guidance; and the new capability-and-limits section.

The last two are the systems story.

Under “Be clear about capabilities and limits”, the assistant should help users maintain an accurate mental model of what it can and cannot do in the current context. The text explicitly warns that tool access may be powerful but incomplete, and that capabilities can change across model versions, application settings, and contexts. Its examples are concrete: do not claim to have permanently forgotten information when the system cannot erase chat history; do not pretend to revise a missing document.

The update also says that when a request contains a false or potentially false premise, the assistant should briefly flag the mismatch rather than silently building on it. The full specification asks for calibrated framing: concise, neutral, non-patronising, and not repeated after mutual understanding is established.

Observed fact: OpenAI has now specified that capability honesty and premise correction are expected model behaviours.

Important qualifier: “Be clear about capabilities and limits” is labelled a Guideline, the lowest authority tier in the document. The Model Spec says guidelines can be overridden implicitly by contextual cues, while User, Developer, System, and Root instructions sit above them in that order. The broader “Do not lie” rule is User-level, but the new proactive duty to maintain the user’s mental model is not a hard runtime invariant.

The spec says it is not the deployed system

OpenAI is unusually explicit about the boundary. In the overview to the August 18 specification, it says the public version may omit details and that production models do not yet fully reflect the Model Spec.

That sentence should govern how operators read the rest of the document. The Model Spec is a statement of intended behaviour and a training target. It is not:

a cryptographic record of the model and policy snapshot that handled a request;
a capability manifest for the tools available in that session;
proof that a tool call reached its destination;
proof that the external system accepted or persisted a mutation;
a receipt tying the model’s final sentence to the observed result.

This is not a criticism of publishing a spec. It is a category boundary. A source-code style guide does not prove a binary was compiled from that source; a policy document does not prove a production request followed it. Inference: the more authority agents receive, the less safe it becomes to treat fluent self-reporting as evidence of execution.

Why tool use creates an attestation problem

The same document already contains the ingredients of the failure.

The Model Spec defines a tool as a program an assistant can call, notes that tools may create difficult-to-reverse side effects, and says agentic contexts require extra care. Its Root-level side-effect rule tells assistants to minimise bad surprises, prefer reversible approaches, back up state, conduct dry runs, and document actions.

It also classifies tool output as untrusted by default. Instructions inside tool results have no authority unless authority is delegated to them. The assistant is expected to reason about the tool, endpoint, relevance, trust level, and possible side effects.

That creates at least four separate states which chat interfaces often collapse into one word — “done”:

State	What is actually known
Intended	The model formed a plan to perform an action.
Dispatched	The runtime submitted a tool request.
Acknowledged	The tool or remote endpoint returned a success-shaped response.
Verified	Fresh observation confirms the intended state now exists.

A model can be perfectly honest at the first state and still be wrong about the fourth. The tool may time out after applying a mutation. An API may return 200 before an asynchronous job fails. A browser click may land on the wrong control. A deployment command may succeed while the public CDN serves stale content. A malicious tool result may even present a fabricated success message as data.

No amount of instruction tuning removes those distributed-systems conditions.

The missing layer is a claim-to-evidence contract

Agent runtimes need to make action claims machine-checkable. A useful completion contract would bind every consequential claim to an evidence record with at least:

Action identity — a stable ID, tool name, endpoint class, timestamp, and idempotency key where supported.
Authority — which user-approved scope permitted the action, including cost and side-effect limits.
Request digest — a hash or safely redacted canonical form of what was sent.
Tool acknowledgement — raw status, remote object ID, transaction ID, or commit hash rather than a prose summary.
Independent verification — a fresh GET, file read, status query, checksum, or public request that does not merely replay the mutation response.
Final-state vocabulary — planned, sent, accepted, verified, failed, or unknown; never a single ambiguous success boolean.

For a static-site deployment, “verified” might mean: local build exited zero; a server-side rollback archive exists; artifact checksums match; Caddy remains active; the exact public URL returns HTTPS 200; and the returned HTML contains the title, a distinctive paragraph, and a source link. For an email, it might require a provider message ID plus a subsequent sent-folder lookup. For a file edit, it means reading the file back or checking its digest—not trusting the editor’s own success string.

The verifier should also be structurally separate from the actor when stakes justify it. If the same model interprets an ambiguous request, performs a mutation, summarises the tool response, and judges its own success, every layer shares one failure mode. A narrow deterministic check—or a second bounded agent with read-only access—breaks that correlation.

What operators should change now

The August 18 update improves the model-side default. Operators should treat it as a prompt to harden the other side of the interface.

First, expose capabilities as session state. The model should not infer whether it has network access, a persistent memory store, deployment credentials, or permission to send messages. Give it a runtime-generated capability manifest with expiry and scope.

Second, make mutation tools return handles, not adjectives. “Success” is weak. A commit SHA, deployment ID, message permalink, object version, or absolute path can be checked.

Third, require read-after-write verification for consequential actions. A successful POST is evidence that a request was accepted, not that the desired state exists. Verification should query the destination through the most direct available path.

Fourth, preserve uncertainty. If the runtime cannot observe the final state, the correct result is unknown, not an optimistic paraphrase. This is where the new Model Spec guidance helps: a model trained to maintain an accurate capability picture is more likely to communicate that boundary clearly.

Fifth, log the evidence chain. Users should be able to inspect what happened without requesting hidden reasoning. Tool calls, approvals, status transitions, identifiers, and verifier results are operational facts; chain-of-thought is not required to audit them.

The uncomfortable conclusion

OpenAI’s new wording is directionally right. Agents should correct false premises. They should not claim capabilities they do not have. They should tell users when memory, tools, or context differ from what a human assistant would imply.

But the significant change is not that a model vendor wrote down “do not overclaim.” It is that the public specification now names the user’s mental model of capability as something the assistant must actively maintain—while simultaneously acknowledging that deployed models do not yet fully implement the spec.

That gap cannot be closed by the model alone. A trustworthy agent needs a behavioural policy, a scoped capability system, durable tool receipts, and an independent verifier. The policy tells the model what it should say. The evidence tells the user what actually happened.

A claim is language. An attestation is infrastructure.