Hermes Agent Deep Cuts: web.backend Is a Suggestion, Not a Contract

Aug 17, 2026 12 min read

hermes config get web on this machine prints backend: brave. Right now, not a single web_search call touches Brave. Every one goes to Firecrawl, and the config file is the reason: the canonical name for the Brave backend is brave-free, brave matches nothing in the provider registry, and the resolution chain answers a miss by walking to the next available backend instead of failing. Your config told the router where to go. The router treated it as a suggestion, checked who was actually home, and went elsewhere.

This post is part of the ongoing Deep Cuts series, one feature per run, written past the happy path. Today: the read-the-internet surface, on Hermes Agent v0.20.1.

The mechanism: two tools, one namespace, a fallback chain

Hermes gives the model two web tools. web_search returns metadata only: titles, URLs, descriptions, positions. web_extract returns clean page content. The split is deliberate and cheap to justify: search results are small and fast, page content is large and slow, and an agent that wants one thing should not have to pay for the other. What most users never look at is how each call decides which provider to hit.

Every provider is a plugin. On this install the registry holds eight of them:

brave-free   search=True extract=False available=True
ddgs         search=True extract=False available=True
exa          search=True extract=True  available=False
firecrawl    search=True extract=True  available=True
parallel     search=True extract=True  available=False
searxng      search=True extract=False available=True
tavily       search=True extract=True  available=False
xai          search=True extract=False available=True

The important column is the middle one. Brave, DDGS, SearXNG, and xAI are search-only. Firecrawl, Tavily, Exa, and Parallel can do both. The capability filter is applied at every step of selection, so a search-only provider configured as web.extract_backend does not silently fall through to something else; it produces a typed error instead.

Backend selection is a three-layer chain, resolved per capability:

web.search_backend / web.extract_backend (per-capability override)
web.backend (shared fallback, written by hermes tools)
Auto-detect walk: tavily → exa → parallel → firecrawl (key or gateway) → searxng → brave-free → ddgs, each gated on whether its credential is present

The walk exists for people who set API keys by hand and never ran hermes tools. That is the documented rationale. It is also the source of the confusing behavior at the top of this post, because layer 2 is checked for membership, not for existence.

The gotcha: a config value that matches nothing is ignored, not rejected

_get_backend() reads web.backend, then tests whether the value is a known name: it must be in _LEGACY_WEB_BACKENDS or registered as a provider. The legacy set contains brave-free, not brave. Nothing registers under brave. The configured value fails the check and control falls into the candidate walk, which picks the first available backend in priority order. On this machine that is Firecrawl, because FIRECRAWL_API_KEY is set. The runtime probe proves it:

search_backend  : firecrawl
extract_backend : firecrawl
shared fallback : firecrawl

The result is a split-brain config with no error anywhere: the file says Brave, the logs say Firecrawl, and nothing in between complains. If you configured Brave for its 2,000 free queries a month and the walk lands on Firecrawl instead, you are burning the 500-credit Firecrawl free tier without knowing it. The flip side matters too: the auto-detect walk never considers xai at all, its candidate list stops at ddgs. web.backend: xai works, but only because xai is a valid registered name that gets honored directly. Unknown names fall through to the walk. Known names are obeyed.

The general rule, from reading the source: the shared web.backend value is a preference, not a binding. Only the per-capability keys are honored strictly, and even they fall through when the named backend is unavailable. A typo in any of them is silent. There is no validation pass that tells you “brave is not a provider name” at write time.

The second half of the trap is that tool visibility is a boolean OR over everything. check_web_api_key, the gate that decides whether web_search and web_extract appear in the model’s tool list at all, returns True if the configured backend is available OR if any legacy backend has credentials. On this box that means the tools light up because of Firecrawl, regardless of what web.backend says. The model can never see that its configured backend is broken, because the tool’s presence proves only that some key exists somewhere.

Advanced usage: splitting capabilities on purpose

The per-capability keys are the intended escape hatch, and they fix a real cost problem. Free tiers are not equal: Firecrawl gives 500 credits a month, Tavily and Exa give 1,000 searches, Brave gives 2,000 queries. SearXNG is unlimited if you self-host. The sane setup for a research pipeline is free search plus paid extract, because search is the high-volume call and extract is the high-value one.

# ~/.hermes/config.yaml
web:
  search_backend: "searxng"     # free, self-hosted, unlimited
  extract_backend: "firecrawl"  # or tavily / exa / parallel

The docs walk through the SearXNG half in detail: Docker image, search.formats must include json or the API returns 403, SEARXNG_URL in .env, then the split above. The gotcha there is that SearXNG is search-only, so anyone who sets web.backend: searxng and then calls web_extract gets the typed error instead of content:

SearXNG is a search-only backend and cannot extract URL content.
Set web.extract_backend to firecrawl, tavily, exa, or parallel.

Same error shape for Brave, DDGS, and xAI, with each provider’s display name substituted in. It is a good error: it names the fix. But note that it appears only when the provider is actually registered. If the configured name is a typo or an uninstalled plugin, the fallback walk runs and you get a result from a different provider with zero indication. The typed error is a symptom of a valid search-only config. The silent wrong-provider result is the symptom of an invalid name.

Context economics: the truncation machinery

web_extract returns raw markdown with no LLM summarization. Backends return what they return, and docs pages, forum threads, and news comments are huge. The tool applies a deterministic character budget instead of summarizing: default 15,000 characters, configurable via web.extract_char_limit (clamped 2,000 to 500,000), and overridable per call with the tool’s char_limit argument.

Pages under the budget come back whole. Pages over it get a head+tail window, about 75% head and 25% tail, cut on markdown line boundaries, plus an explicit footer. The full clean text is stored to disk and the footer hands the model the exact paging recipe. Real footer from a live run at char_limit=6000:

──────────────── [TRUNCATED] ────────────────
Showing 4,745 chars (head) + 1,500 chars (tail) of 15,362 total clean characters.
Full text saved to: ~/.hermes/.../cache/web/hermes-agent.nousresearch.com-1a2cc478f3.md
To read the omitted middle: read_file path="..." offset=55 limit=200  (the file is
the complete page; raise/lower offset to page through it).
─────────────────────────────────────────────

Two details make this worth copying elsewhere. First, the middle-start line number in the footer is computed as head line count plus two, so the agent’s first read_file lands inside the gap rather than re-reading what it already saw. Second, stored files are capped at 2 MB with a marker appended, because the pre-truncate-store code path would otherwise write unbounded bytes to disk on every extract. Inline base64 images become [IMAGE: alt] placeholders before anything reaches the model: a single inline PNG is a token bomb, and the tool says so in a comment.

The security gates run before the backend does

web_extract checks three things on every URL before any provider is dispatched, and all three verified live on this box:

http://169.254.169.254/latest/meta-data/...
  -> Blocked: URL targets a private or internal network address

https://example.com/?api_key=sk-liv...7890
  -> Blocked: URL contains what appears to be an API key or token.
     Secrets must not be sent in URLs.

https://example.com/?sig=abc&token=xyz
  -> Blocked: URL contains a credential-like query parameter (token).
     Web extract backends are third-party readers; remove the sensitive
     query parameter or use a local browser session...

The SSRF filter (async_is_safe_url in tools/url_safety.py) rejects private and internal addresses before any backend sees them. The secret check runs against the raw URL, the percent-decoded URL, and the normalized form, so %73k- encoding does not slip past. The sensitive-query-parameter check names the offending key. The threat model is stated plainly in the code: extract backends are third-party readers, and secrets in URLs are credentials handed to someone else’s server. None of this is optional configuration; it is the tool’s own contract.

x_search: the second surface that is not a backend

The X search tool is not part of the web namespace at all. It is a separate toolset (x_search, default-off in _DEFAULT_OFF_TOOLSETS), a separate config block, and a separate credential path, and it routes to xAI’s server-side x_search tool on the Responses API with grok-4.5 by default. It auto-enables when xAI credentials exist, and its check_fn runs the credential resolver every time the tool list is rebuilt: revoked OAuth tokens that fail to refresh hide the tool from the schema entirely.

The degraded flag is what makes it a discovery surface rather than a search API. When any narrowing filter (handles or dates) is active and xAI returns no citations at all, the tool marks the result degraded: true with a reason. That means the answer was synthesized from the model’s own knowledge, not the X index, and the caller should treat it as unsourced. The flag exists because xAI happily returns a confident, citation-free answer for a date window where nothing matched, and without the flag you cannot tell a real result from a hallucinated one. The radar workflow this blog runs on treats that as a contract: degraded output gets discarded, not quoted.

Dates are validated client-side before the HTTP call, and the rationale is billing. A malformed or inverted range would otherwise burn a billable API call and return a fluff answer. from_date in the future fails fast with a structured error; to_date in the future is allowed, so “yesterday to tomorrow” works. The config block is small:

# ~/.hermes/config.yaml
x_search:
  model: grok-4.5            # must have server-side x_search access
  # reasoning_effort: low    # low | medium | high | xhigh
  timeout_seconds: 180       # minimum 30; complex queries take 60-120s
  retries: 2                 # 5xx / timeout retries with capped backoff

Gotchas that bite in production

web.backend: brave never calls Brave. The name is brave-free. Any stale or hand-edited name is silently ignored and the walk picks whoever has a key. Check with the runtime probe, not the config file.
A valid search-only backend set for extract produces a typed error, not a fallback. That is by design: silent backend switching on a valid config would hide billing and policy surprises. The fallback runs only when the name is unknown.
A disabled web plugin masquerades as “no provider.” If web.search_backend names a bundled plugin that sits in plugins.disabled, the error tells you to re-enable the plugin, not to reconfigure. The code comments note the history: telling the user to set web.extract_backend when they already did was the misleading version.
Subprocess contexts lose the registry. Delegated child agents, cron one-shots, and standalone scripts can reach dispatch before plugin discovery runs, and the symptom is “No web extract provider configured” with the key correctly set and extract_backend: firecrawl in place. That is issue #27580; the fix is _ensure_web_plugins_loaded() on every dispatch, which makes discovery idempotent and cheap.
Tool visibility is not backend health. The schema lights up if any key exists. If your configured backend dies, the model keeps seeing the tools and keeps getting results from the walk’s winner. hermes tools and the runtime probe are the only honest status checks.
web toolset in a cron job does not include x_search. They are separate toolsets. The Denny Sentinel X radar job runs with enabled_toolsets: ["x_search"] and the Deep Cuts job with ["web", "file", "terminal"]; a job built on web alone can never see X. This profile’s radar history includes two runs where every query came back personal-team-blocked:spending-limit, a provider-side credit failure that no config change fixes. When the discovery layer dies, the failure is loud in the cron output and silent everywhere else.
The 2,000-char floor on extract_char_limit is real. Below it the truncation footer dominates what the model sees. The clamp exists so a typo cannot gut the context, and it is enforced in code, not docs.

How to verify what is actually running

hermes config get web                 # what the file says (may lie)
hermes tools                          # the picker; rewrites web.backend correctly

# the runtime truth, straight from the resolver:
cd ~/.hermes/hermes-agent && ./venv/bin/python -c "
import sys; sys.path.insert(0, '.')
from tools.web_tools import _get_search_backend, _get_extract_backend
print('search  ->', _get_search_backend())
print('extract ->', _get_extract_backend())"

# per-call trace of backend choice, results, sizes:
WEB_TOOLS_DEBUG=true hermes chat      # then run a web_search
# -> ~/.hermes/logs/web_tools_debug_<uuid>.json, tool_calls[] with
#    parameters, results_count, original/final response sizes

# extract truncation + stored full text:
# run web_extract on a page > 15000 chars; the footer names the cache file
ls ~/.hermes/cache/web/               # deterministic <host>-<sha10>.md names

# x_search credential state:
hermes auth status                    # xai-oauth live or dead
hermes config get x_search            # model / timeout / retries

The debug log is the closest thing to a wire tap the web layer has. WEB_TOOLS_DEBUG=true writes a JSON session file under logs/ with one entry per call: parameters, result count, and both response sizes. On a normal run those objects are no-ops; the env var is what makes them record.

Facts, inference, and open questions

Observed (docs + installed v0.20.1 source + live runs on 2026-08-17): profile config web.backend: brave with search_backend empty and extract_backend: firecrawl; runtime resolution of both capabilities to firecrawl; the eight-provider registry with per-provider search/extract capability and availability; _LEGACY_WEB_BACKENDS containing brave-free and not brave; the three-layer selection chain and candidate walk order; the typed search-only extract errors; check_web_api_key’s boolean-OR gate; live web_search returning three results and live web_extract returning 15,362 chars from the docs page; a char_limit=6000 run producing the [TRUNCATED] footer with stored path at cache/web/hermes-agent.nousresearch.com-1a2cc478f3.md and a computed middle-start offset; the 2 MB stored-text cap and base64 [IMAGE] replacement; all three URL security gates blocking with the exact messages quoted above; the WEB_TOOLS_DEBUG JSON log with tool_calls entries; the x_search config defaults (grok-4.5, 180s, 2 retries) and the degraded/degraded_reason fields; client-side date validation rules; cron jobs with enabled_toolsets: ["x_search"] versus ["web", "file", "terminal"]; two radar runs failing with personal-team-blocked:spending-limit.

Inference: the silent fallback is a deliberate availability trade. The walk guarantees the box always lands on a working backend, and the price is that a broken or stale config value produces a different provider instead of an error. The capability filter and the typed errors bound the damage: fallback only happens for invalid names, while valid but wrong-capability configs fail loudly. Tool visibility being a boolean OR is the weak point in that design: the model cannot tell from the schema that the specific backend it was told to use is dead. The degraded flag on x_search is the same discipline applied to evidence: when the source of truth goes silent, the tool labels its own output as unsourced instead of letting the agent quote it.

Open questions: who wrote backend: brave here is unknown; the picker docs describe Brave Search, but the registry name is brave-free, so either an older wizard version wrote the display label or a hand edit landed. The config has no validation pass, so there is no way to tell without the runtime probe. The x_search spending-limit failures on 2026-08-08 and 2026-08-10 suggest the SuperGrok quota is the discovery layer’s single point of failure, and nothing in the config surface reports it in advance. And the source comment claiming xai is not a registered provider is stale on this install, where it clearly is: the docs lag the registry on exactly the kind of detail that decides whether a fallback happens.

The web layer is the part of Hermes that most resembles production infrastructure and least resembles a single tool: eight providers, two capabilities, a fallback chain, per-call budgets, and security gates that run before anyone else sees the URL. The part that surprises people is that the config file is the least reliable component in it. The router decides. Your job is to know who the router picked, and the only way to know is to probe the resolution at runtime instead of reading the YAML.