Why Security Agents Need a Code Map Before a Bigger Model
A bug-hunting agent can spend most of its budget reading the code before it ever tries to trigger the bug.
That is the finding worth keeping from a new paper on the cost of LLM agents for vulnerability discovery. Across 200 CyberGym traces from four agents, code localization, code understanding, vulnerability reasoning, and trigger design accounted for 60.4% of tokens. In failed runs, those activities were also the leading bottlenecks.
The uncomfortable truth is that many security agents are not primarily search systems. They are very expensive code readers that repeatedly rebuild the same map.
The claim and the evidence
The paper, Where Do the Tokens Go? Understanding and Reducing Costs in LLM Agents for Vulnerability Discovery, was submitted to arXiv on October 8, 2026. Its authors studied 200 traces spanning Codex, OpenCode, Cybench, and EnIGMA, comparing an unaided baseline with four existing efficiency methods.
The results are more useful than a generic claim that agents are expensive:
| Finding | What it changes |
|---|---|
| 60.4% of tokens went to localization, understanding, vulnerability reasoning, and trigger design | The first optimization target is the investigation loop, not only generation |
| Only 24.4% of matched comparisons preserved success at lower total cost | A cost reduction can quietly become a detection reduction |
| AVRI reduced total cost by 18.0% for Codex and 23.7% for OpenCode on 20 tasks | Reusable evidence can outperform generic efficiency tricks |
These are the paper’s reported results, not an independent reproduction. A post from Rohan Paul circulated the headline numbers on October 19 and described the study as a warning against applying cost cuts before measuring where tokens go.
The repeated work is the architecture problem
A conventional vulnerability-discovery loop often looks like this:
read repository
find suspicious code
reason about reachability
design an input
run the target
inspect the result
repeat
The loop is sensible. The waste appears when every attempt starts by reconstructing the same facts about the repository.
Which function consumes the input? Which parser transforms it? Which branch reaches the unsafe operation? What assumptions does the harness make? Which parts are confirmed by source evidence, and which are still hypotheses?
If the agent does not preserve those relationships in a durable form, the next attempt pays to discover them again. A larger context window does not fix that. It only makes repeated reconstruction more expensive when the agent fills the window with another broad slice of the repository.
AVRI makes evidence reusable
The paper’s proposed interface is AVRI, an Agent-centric Vulnerability Reasoning Interface. Its central data structure is a persistent Bidirectional Evidence Trace, or BET.
BET connects two directions of the investigation:
program input -> harness behavior -> selected operation -> unsafe condition
The trace retains source-supported correspondences alongside the agent’s hypotheses and open questions. The interface then gives the agent reading, analysis, and persistence commands to build and reuse that evidence.
That distinction matters. A summary says what the agent thinks it learned. An evidence trace records why the agent thinks it learned it, which source locations support the claim, and what remains unproven.
This is closer to a working research notebook than to a larger prompt. The artifact is not just context. It is a map with provenance.
Why a cheaper model is not the first answer
The obvious response to a large token bill is model routing. Send repository reconnaissance to a smaller model, reserve the expensive model for the final reasoning pass, and reduce the price per token.
That can help. It can also optimize the wrong layer.
If the agent repeatedly reads irrelevant files, a cheaper model still spends tokens on irrelevant files. If the trigger path is not represented explicitly, each model must infer it again. If the evidence is not persisted, the system pays the same analysis cost on every attempt.
The paper reports that only 24.4% of matched comparisons preserved success at lower total cost across the tested methods. That is a warning about the verifier, not only the model bill. A security agent is useful when it produces a reproducible, accepted finding. A cheaper run that produces a plausible but untriggered hypothesis has not reduced the cost of discovery. It has moved the cost into triage.
The better routing boundary is therefore often structural:
- Use cheap parsing and indexing to build repository facts.
- Use a focused analysis pass to connect input, reachability, and unsafe operations.
- Use a stronger model where the evidence map leaves a real reasoning question.
- Preserve the map so the next attempt does not start from zero.
- Require an executable or otherwise auditable verifier before calling the result a finding.
The model can still change between stages. The important change is that every stage receives a bounded, relevant, source-linked problem.
The security boundary is the evidence contract
Persistent context is not automatically trustworthy context.
An agent that stores its own conclusions can also store its own mistakes. A wrong reachability assumption may become a durable false premise and contaminate later attempts. The trace therefore needs more than text fields. It needs provenance, versioning, and invalidation rules.
Operators should ask at least four questions:
- Which repository revision produced this evidence?
- Which source locations support each relationship?
- Which items are observed facts, and which are hypotheses?
- What event invalidates the trace, such as a code change or harness change?
The same rule applies to model-generated maps as to human notes. Reuse is valuable only when the system can tell old evidence from current evidence.
What builders should measure next
Token totals are a useful start, but they are not the production metric. A security agent should report cost per reproducible finding, not only cost per run.
That measurement should include:
- tokens spent on localization and code understanding
- tokens spent on trigger reasoning
- percentage of traces that reuse prior evidence
- success rate after each cost reduction
- cost per finding that survives analyst review
- time spent by a human rejecting unsupported reports
Those numbers expose whether an optimization improves the control loop or merely makes its failures cheaper.
The paper’s result points to a practical design principle: investigate once, preserve the evidence, and spend model capacity on the unresolved edge of the problem.
A security agent does not need a bigger memory of everything. It needs a smaller, versioned map of what matters.