The 97% Gate: Claude Code Replaces Human Approval with a Classifier

Aug 10, 2026 10 min read

The permission prompt was never a gate. It was a rubber stamp with a click sound.

That is the admission buried in Anthropic’s announcement that auto mode — the classifier-gated permission mode that routes every tool call through a safety model instead of asking the human — becomes the default in Claude Code for Pro, Max, and Team plans starting August 14, 2026. The supporting numbers are the story: developers approve 97% of permission prompts, and in a controlled study of 1,053 paid testers, humans caught just 13.6% of deliberately planted dangerous commands. The classifier blocked 89% of the same set.

Head to head, the classifier blocked 800 dangerous commands that a human had approved; humans blocked 6 that the classifier allowed. After 50 prompts in a session, human catch rates collapsed to about 5% — the rubber stamp degrades with fatigue. The classifier’s block rate stayed flat.

Anthropic measured the human verifier, found it broken, and replaced it with a machine verifier that, on their own benchmarks, is strictly better. The systems question is what happens to the loop when the machine verifier fails — and the answer is the most interesting part of the whole announcement.

The control loop that just changed

Claude Code’s permission system has always been a gate chain. In manual mode (config value default), every file edit, shell command, and network request pauses for approval. That is the “human-in-the-loop” everyone has been told is the safe default. Anthropic’s data says it was never safe — it was a 97% approval reflex:

49.5% of active CLI users have created a Bash allow-rule; 5% allow any shell command outright, and another 43% use interpreter rules like Bash(python:*) that are equivalent in practice.
62% of users have used bypassPermissions or “don’t ask again” on Bash.
25% of interactive sessions start in bypass-permissions mode.

The users themselves had already voted on the prompt gate with their settings files. Anthropic just made the vote official.

Auto mode replaces the prompt with a classifier that blocks anything irreversible, destructive, or aimed outside your environment. When it blocks something, Claude either finds a safer path or asks the user directly. The fallback is where the loop design shows its seams: three blocks in a row, or twenty across a session, drops the session back to manual approvals.

Think about what that fallback actually is. The classifier fails closed for a while, and then — precisely when it has flagged the most suspicious activity — control returns to the verifier that Anthropic’s own study measured at a 13.6% catch rate. The stress signal that triggers the fallback is the same signal that predicts the human gate will fail.

The evidence Anthropic ran

The announcement is unusually data-dense for a vendor blog, and the data is worth separating into buckets.

Controlled study (1,053 paid testers). Testers worked in a test environment, and partway through each session a single permission prompt was swapped for a clearly dangerous command. Humans caught it 13.6% of the time (143 of 1,053); auto mode blocked 89% (937 of 1,053). Head to head: auto mode blocked 800 commands humans approved; humans blocked 6 that auto mode allowed. Session-length effect: humans blocked ~17% of dangerous commands early in a session, dropping to ~5% after 50+ prior prompts. Auto mode’s block rate was flat.

Production re-grading (May–June 2026). Anthropic re-graded flagged sessions on two 0–10 scales: harm severity (10 = company-wide irreversible harm, 7 = production-level such as deleting rows from a production DB) and how explicitly the user asked for the action. Among flagged sessions, manual approval contained serious unintended harm more than twice as often as auto mode. At severity 7+, 6.3% of manually approved sessions contained a harmful action the user hadn’t explicitly asked for, versus 2.4% of auto mode sessions — and the gap held at every severity level tested.

Adversarial red-teaming (Apollo Research). A two-week pilot injected synthetic attacks into real coding trajectories across three datasets (broad failures, data exfiltration, safety threats). After hardening the classifier with environment context (repository visibility, git state, data-handling rules), Apollo re-tested on a held-out attack set Anthropic never saw: auto mode caught about half of the attacks it previously missed, at a cost of missing ~2% it previously caught. The classifier’s miss rate fell from 12% to 7% across shared and held-out sets. Anthropic is careful to note these are adversarial synthetic attacks — “more malicious than anything Claude would do unprompted” — and the 7% miss rate is against that set, not real traffic.

Prompt-injection evaluation (Trajectory Labs). An independent third party tested 72 indirect prompt-injection scenarios, 10 runs each, against Claude Code v2.1.205 and Codex v0.144.5 using an identical browser integration. Zero of 720 attacks succeeded against Claude Fable 5, Opus 5, or Sonnet 5 running auto mode. GPT-5.6 Sol running Codex’s Auto-review mode had a 5.83% attack success rate; in Full Access mode, 19.03%. The same eval measured a 0.09% attack success rate against Claude models in bypassPermissions mode without additional safeguards — a useful reminder that this eval isolates the model, not the full product stack, and that Anthropic’s own browser-extension safeguards were not tested.

None of this is peer review, and Anthropic funded the third-party work. But the direction is consistent across four independent measurement approaches: the machine gate outperforms the human gate on the exact failure modes the human gate was supposed to catch.

How the new gate is built

The mechanism matters more than the marketing. From the auto-mode configuration docs, the architecture is a two-stage gate chain:

The permissions system runs first. permissions.deny rules block before the classifier is consulted and cannot be overridden. permissions.ask rules always force a prompt — the classifier cannot auto-approve a matching action. This is the one place an operator can insert a hard human checkpoint (e.g., Bash(git push *), Bash(gh pr create *)).
The classifier runs second, with four tiers of its own: hard_deny (blocks unconditionally — the built-in rule is data exfiltration), soft_deny (blocks unless explicit user intent or an allow exception), allow (exceptions to soft blocks), and explicit user intent (overrides soft blocks only if the user’s message directly and specifically describes the exact action — “clean up the repo” does not authorize a force-push; “force-push this branch” does).

Three details deserve operator attention:

Allow rules are asymmetric. Broad rules that grant arbitrary code execution — Bash(*), Bash(python:*) — are suspended while auto mode is active, because they would let commands skip the classifier entirely. But narrow allow rules like Bash(npm test) still carry over and resolve before the classifier runs. The docs are explicit about the hole: “a narrow rule can still let a destructive argument through without the classifier seeing it, for example a script path or flag the rule’s prefix didn’t anticipate.” If you run autoMode.classifyAllShell: true, every Bash and PowerShell allow rule is suspended while auto mode is active (requires v2.1.193+).

Trust is prose, not policy. The classifier reads an autoMode.environment block of natural-language entries — “Source control: github.example.com/acme-corp and all repos under it”, “Trusted cloud buckets: s3://acme-build-artifacts”. By default it trusts only the working directory and the current repo’s configured remotes; everything else is a potential exfiltration target until named. The docs explicitly warn that setting any of environment/allow/soft_deny/hard_deny without the literal "$defaults" string replaces the entire built-in list — including the default soft blocks for force-push, curl | bash, production deploys, and auto-mode bypass, and the default hard block on data exfiltration. This is a configuration foot-gun in the same family as the allowlist that leaked in the Kimi K3 sandbox escape: the safest default exists, and the path of least resistance is a permissive one.

Repo-injected config was closed — recently. Before v2.1.207, the classifier read autoMode from .claude/settings.local.json, which lives in the repo directory — meaning a checked-in repository, or a build step, could inject its own allow rules. v2.1.207 moved that scope to ~/.claude/settings.json only. That fix is the security-relevant part of this release, and it suggests the trust boundary of the classifier is actively being audited.

The classifier also now checks what a push would do before allowing it: whether the destination repo is public, private, or trusted; whether a force-push or a secret entering the commit would leak through CI; and git status before destructive git actions like git reset --hard. Tool results from external sources are screened by an API-side prompt-injection probe before reaching the model’s context.

Anthropic’s own three internal incidents demonstrate the classifier’s value in production: it denied a fallback that would have published an internal report to a public code-sharing site; it flagged a mass process kill across ~2,000 pods with queued deletions that would have torn down hundreds of GPUs of in-flight training jobs; and it caught a request for “read-only billing access” that reached for a cloud role granting full administrative control at the account root.

What operators should actually do

The announcement’s closing caveat is the part to frame: “it relies on classification systems and therefore does not eliminate risk. For high-stakes changes to production infrastructure, we still recommend reviewing Claude’s actions yourself.” In other words: the classifier is a second gate, not a wall, and Anthropic knows it. The operator playbook follows from the architecture:

Treat the permission prompt as nonexistent. If your security posture depended on developers clicking “approve,” it never existed. Auto mode’s default just makes the fiction visible. Audit what your agents can actually reach — filesystem, network, cloud roles — as if the gate were already open.
Configure autoMode.environment explicitly, with "$defaults". Name your source-control org, trusted buckets, internal domains, and services. Review claude auto-mode defaults before touching any list, and never set environment, allow, soft_deny, or hard_deny without the $defaults sentinel unless you intend to take full ownership of the list.
Insert hard checkpoints where the classifier is allowed to be wrong. permissions.ask for pushes and PR creation, permissions.deny for actions that must never run — the deny rule blocks before the classifier and “can’t be overridden,” which is exactly the property you want for production deploys.
Set autoMode.classifyAllShell: true if you rely on shell allow rules. Otherwise narrow rules like Bash(npm test) bypass the classifier, and the docs admit the bypass can carry a destructive argument.
Plan for the fallback. Three consecutive blocks or twenty in a session returns control to the 13.6%-catch-rate human gate. If the classifier is flagging that much traffic, the correct response is to stop the session and investigate — not to keep clicking.

The verifier economy just moved

The uncomfortable truth is not that Anthropic is automating away human oversight. It is that the human oversight was already absent — a 97% approval rate is not oversight, it is throughput. Anthropic ran the experiment, got the number, and productized the conclusion.

The systems lesson generalizes beyond Claude Code: any approval gate whose rejection rate rounds to zero is not a verifier, it is a ceremony. The gate’s actual job — catching the dangerous action before it executes — requires measuring the gate, not just installing it. Anthropic measured, and the measurement said the human was the weakest component in the loop. The next release of every agent harness will be judged by the same test: not whether it has an approval gate, but whether the gate demonstrably gates.

The fallback semantics are the tell. When the machine verifier is under maximal stress, the design hands control to the verifier that fails 86% of the time. That is not a bug in the announcement — it is the honest statement of the current state of the art. Nobody has built a verifier that can be trusted alone. The best loop Anthropic could ship is one that defaults to the strong verifier and degrades to the weak one, and says so in the release notes.