feat(web): configurable fallback chain for web_extract

[victory-c]

This PR implements a fallback chain for web_extract_tool, addressing Issue #68516.

Problem

Previously, web_extract relied on a single backend. If that backend became unavailable (API credits expired, timeouts, rate limits), the extraction failed entirely, requiring manual configuration changes and a gateway restart.

Solution

Users can now configure an array of fallback providers:

web:
  extract_backends:
    - firecrawl
    - direct
    - browser

The dispatcher iterates through these configured backends. If a backend fails or returns an empty response, it logs a warning and seamlessly attempts the next provider in the chain. It only surfaces an error if all configured backends fail.

Legacy backward compatibility is fully preserved: if only web.extract_backend or web.backend is set, the system behaves exactly as it did before.

Closes #68516.

[alt-glitch]

This was generated by AI during triage.

Duplicate of #23366: both PRs implement configurable web_extract provider fallback chains in tools/web_tools.py, including retry after extraction failures. #23366 is the earlier open implementation.

[teknium1]

Thanks for adapting the fallback idea to the current provider registry. The single-backend premise remains valid on current main (tools/web_tools.py:858-944), but the fallback dispatch needs correction before it can preserve registry behavior.

Problems

• tools/web_tools.py:872 resolves and availability-filters the chain before plugin discovery at line 874. This can drop a custom fallback in a cold-start process; main already requires discovery before extract registry lookup (tests/tools/test_web_providers.py:350-435).
• tools/web_tools.py:902 substitutes the scalar active provider for an unregistered configured chain entry. The active resolver only reads web.extract_backend / web.backend (agent/web_search_registry.py:291-298), so this can silently dispatch outside the configured chain.
• tools/web_tools.py:953 compares names with the final entry instead of tracking position; duplicate entries can stop a chain before a remaining distinct fallback is attempted.

Suggested changes

• Discover plugins before chain resolution, resolve explicit entries exactly, and deduplicate or index the chain.
• Add dispatch tests for cold-start custom fallbacks, all-error/empty/exception outcomes, and duplicate entries.

Automated hermes-sweeper review.

[teknium1]

_get_extract_backends() filters via _is_backend_available(), which consults the provider registry, but discovery happens on the next line. In a cold-start process, an available built-in primary can cause a custom plugin fallback to be removed before that plugin is registered. Please run _ensure_web_plugins_loaded() before resolving the chain and cover this mixed built-in/custom case.

[teknium1]

For an explicit fallback list, an unregistered entry should fail/skip and continue to the next configured entry. Falling back to get_active_extract_provider() here reads only the scalar config keys, so it can invoke an unrelated provider that was not in extract_backends.

[teknium1]

This is not a reliable "last attempt" test when the configured list contains duplicates: ["firecrawl", "tavily", "firecrawl"] stops after the first Firecrawl all-error result and never reaches Tavily. Deduplicate during normalization or use the loop index to decide whether entries remain.

[victory-c]

Thanks for the review, @teknium1 — all three points are addressed in 6b5e5451 ("harden extract fallback dispatch"), and the branch is now merged up to current main (the tests/tools/test_web_providers.py conflict from the test-prune wave is resolved, so the PR is mergeable again).

1. Discovery before chain resolution. _ensure_web_plugins_loaded() now runs before _get_extract_backends() resolves/filters the chain. On a cold-start process a custom plugin fallback that only registers at discovery time is no longer dropped by the availability filter.

2. Explicit chain entries resolve exactly. The get_active_extract_provider() substitution is removed. An entry that doesn't resolve to a registered extract-capable provider now records a typed error and continues to the next configured backend, so dispatch can never step outside the configured extract_backends onto an unrelated web.extract_backend/web.backend scalar.

3. Duplicate-safe last-attempt check. The "is this the final attempt" test now uses the loop index (idx != len(backends) - 1) instead of comparing names, so a chain like ["firecrawl", "tavily", "firecrawl"] no longer stops after the first Firecrawl all-error result and still reaches Tavily.

New regression coverage in tests/tools/test_web_tools_extract_fallback.py exercises exactly the cases you called out: cold-start discovery ordering, all-error / empty / exception fall-through, and duplicate entries. The web-tools suite is green locally.

Ready for another look whenever you have a chance — thanks again. 🙏

[Darko893]

Affiliation: I maintain hermes-plugin-haunt.

I tested this PR at commit 5d823fc8 with the real published hermes-plugin-haunt 0.1.0 entry point:

• the PR's web provider and fallback tests passed: 31/31;
• cold entry-point discovery loaded the Haunt provider successfully;
• extract_backends: [haunt] returned the expected successful extraction;
• Haunt's typed blocked response was treated as an all-error result and correctly advanced to the next configured provider;
• the compatibility check passed Ruff and ty.

One setup naming detail for third-party users:

plugins:
  enabled:
    - web-haunt
web:
  extract_backends:
    - haunt
    - <next-provider>

web-haunt is the installable plugin entry-point name, while haunt is the provider name used in the extraction chain.

The HTTP transport was mocked, so the test made no live API calls and consumed no credits. From the third-party provider side, the fallback behaviour is working as intended on this head.

[alt-glitch]

This was generated by AI during triage.

Re-triage correction: this is related to #23366 rather than a duplicate. Both add extract fallback chains, but #68524 uses one ordered web.extract_backends contract whereas #23366 uses a separate extract_fallback_backends contract and different provider resolution behavior. Maintainer selection is needed.