feat(extensions): allow constructor kwargs on config-declared middlewares

[wutongyuonce]

Fixes #5311

Why

extensions.middlewares only accepted module.path:ClassName strings and instantiated each class with no arguments. Operator-managed middlewares that need a constructor argument (threshold, header name, etc.) had to wrap a hardcoded subclass.

This is the first slice of the documented MVP gap. Lead-only vs subagent-only lists are unchanged.

What changed

• Each extensions.middlewares entry may be a class-path string (existing zero-argument constructor) or {class, kwargs}.

• load_configured_extension_middlewares passes kwargs to the constructor.

• Blank class paths (string or object) fail at config validation. Unknown fields such as apply_to are rejected. Import/constructor errors still fail at agent creation.

• Docs/examples in README, CONTRIBUTING, config.example.yaml, and the harness middleware/customization guides.

• Frontend UI

• Backend API

• Agents / LangGraph

• Sandbox

• Skills

• Dependencies

• Default behavior change — existing string entries still zero-arg construct

• Docs / tests

Validation

cd backend
.venv/bin/ruff check packages/harness/deerflow/config/extensions_config.py packages/harness/deerflow/agents/middlewares/configured_extensions.py tests/test_configured_extensions.py tests/test_app_config_reload.py
PYTHONPATH=. .venv/bin/python -m pytest tests/test_configured_extensions.py tests/test_app_config_reload.py tests/test_middleware_documentation.py -q

84 passed.

Review

Independent Codex review requested one change: blank string entries were accepted by schema and only failed later at reflection. Config validation now rejects whitespace-only string paths and strips padding on valid strings.

AI assistance

Tool(s) used: Pi coding agent (implementation), Codex (read-only review in a Herdr pane)

How you used it: Implemented from a scoped first-slice plan (schema + loader + tests + docs). Codex reviewed the diff and did not edit files. I applied the blank-string validation finding.

• I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

[willem-bd]

Reviewed at cafdf4b. Nice slice: the union schema keeps existing string entries working, extra="forbid" rejects speculative fields like apply_to, blank entries fail at validation instead of at reflection, and docs across README / CONTRIBUTING / both harness guides (en+zh) / config.example.yaml were updated in the same change. I ran the three touched test files in a head worktree (84 passed) and exercised the new spec path directly. Two suggestions and a nit below; the main one is that free-form kwargs makes the value types an operator's middleware receives depend on which file declares the entry.

[willem-bd]

Because kwargs is dict[str, Any], the type a value reaches the constructor with now depends on which file declares the entry: config.yaml is read with yaml.safe_load, so an unquoted ISO date arrives as datetime.date, while the identical entry in extensions_config.json arrives as str. Verified at head with the same dict through both loaders:

middlewares:
  - class: pkg:Mw
    kwargs:
      cutoff: 2026-01-01   # __init__ sees datetime.date(2026, 1, 1)

Two consequences: (a) a constructor expecting a string silently gets a date depending on whether the operator put this block in config.yaml or extensions_config.json (config.yaml wins in the AppConfig.from_file merge, so the same middleware can behave differently across deployments for no visible reason), and (b) to_file_dict() dumps in python mode, so a non-JSON scalar can sit in kwargs — if such a value ever flows into the json.dump inside atomic_write_extensions_config it raises TypeError: Object of type date is not JSON serializable (reproduced).

Consider normalizing/validating kwargs to JSON-serializable values at config validation (e.g. round-trip the spec through model_dump(mode="json") in a kwargs validator, or reject non-JSON scalars with a clear message), and documenting that kwargs values must be JSON types.

[wutongyuonce]

Fixed in f7d40e8.

ConfiguredMiddlewareSpec now walks kwargs at validation: JSON types are kept as-is; YAML date/datetime/time become ISO strings; NaN/Inf and anything else (bytes, objects, …) fail with a clear JSON types error. After that, a constructor always sees the same types whether the entry came from config.yaml or extensions_config.json, and to_file_dict() + json.dump no longer raise TypeError on a YAML date.

Documented on the field, in config.example.yaml, README, CONTRIBUTING, and the harness guides.

[willem-bd]

Nit: this is the only loader branch with no coverage. test_dict_entry_passes_constructor_kwargs feeds an already-validated ConfiguredMiddlewareSpec, so model_validate(entry) on a raw dict never executes — a cheap test would be load_configured_extension_middlewares(_config({"class": f"{__name__}:RecordingMiddleware", "kwargs": {"max_tool_calls": 2}})), which also pins that a malformed raw dict fails loudly here rather than at agent build time.

[wutongyuonce]

Added. test_raw_dict_entry_passes_constructor_kwargs feeds a raw {class, kwargs} dict into load_configured_extension_middlewares, and test_malformed_raw_dict_fails_at_load pins that an unknown field such as apply_to raises ValidationError at load rather than at agent build.

[willem-bd]

Nit: line 46 still reads "Missing packages, invalid classes, and broken modules fail loudly at agent creation with an actionable import error", while README and the harness/config AGENTS guides were extended to include constructor errors. Worth keeping this operator-facing comment in sync ("...and constructor errors fail loudly"), since this is the file operators actually copy.

[wutongyuonce]

Updated. Import failures still say they fail with an actionable import error; constructor errors are called out separately as failing with the underlying exception. Also noted that kwargs must be JSON types (YAML dates/timestamps are coerced to ISO strings).

[willem-bd]

Re-reviewed at f7d40e8. All three findings from the previous round are resolved and verified by execution in a head worktree: YAML/JSON kwargs parity now holds (an unquoted 2026-01-01 in config.yaml and the same value as a string in extensions_config.json both reach the constructor as str, including inside nested dicts), to_file_dict() output is json.dumps-safe so the atomic_write_extensions_config path can no longer raise TypeError, and the raw-dict loader branch is covered by test_raw_dict_entry_passes_constructor_kwargs + test_malformed_raw_dict_fails_at_load. The config.example.yaml comment now calls out constructor errors separately and documents the JSON-types rule. I ran the three touched test files with ruff 0.15.12 (the version uv.lock pins): 90 passed. One thing left before this can merge — backend make lint is currently red on this branch, details below.

[willem-bd]

CI blocker: ruff check fails here with E501 Line too long (289 > 240) — backend/ruff.toml sets line-length = 240 and make lint runs both ruff check . and ruff format --check ., so backend CI fails on this branch. This line was fine before the description was extended in f7d40e8.

The formatter can't rescue this one: it is a single string literal, so ruff format leaves it at 289 chars. I verified that after running ruff format over both touched files, this is the only surviving E501 — so shortening it (or lifting it into a module-level constant next to _JSON_KWARGS_ERROR, the pattern already used a few lines up) is the actual fix, not a reformat.

[wutongyuonce]

Fixed in 530a940. Split the middlewares Field description into concatenated string literals (same pattern as the kwargs description). Longest physical line is now 230, under the 240 cap. ruff check is clean on this file.

[willem-bd]

Nit: ruff format --check also fails on this file (and on extensions_config.py) — the formatter collapses this 3-line call into one 138-char line (and the call at line 61-63 likewise). make format clears it, and unlike the middlewares description E501 this part is genuinely mechanical: after reformatting, the touched files are clean apart from the E501 flagged separately.

[wutongyuonce]

Fixed in 530a940. Ran ruff format on test_configured_extensions.py and extensions_config.py. ruff format --check is clean on both.

[wutongyuonce]

@willem-bd The remaining make lint items from the last pass are addressed in 530a940:

• E501 on the middlewares Field description — wrapped into concatenated string literals (longest physical line is 230, under the 240 cap); ruff check is clean on that file
• ruff format on test_configured_extensions.py and extensions_config.py — ruff format --check is clean on both

[willem-bd]

Reviewed the latest head (08907d9). Two findings below: nested constructor kwargs are shared across middleware instances, and the guidance expansion fails AG002. Validation: 102 focused tests passed; Ruff lint and formatting passed for the changed Python files.

[willem-bd]

[P2] Deep-copy constructor kwargs for each middleware instance

dict(entry.kwargs) only copies the outer mapping. Nested lists and dictionaries remain shared with the cached configuration and every middleware built from it. I reproduced this with a validated ExtensionsConfig containing kwargs: {options: {labels: [configured]}} and a middleware that retains options: constructing two instances and appending to the first instance's labels also changes the second instance and the cached config. Middleware that retains or normalizes mutable constructor arguments can therefore affect other runs or subagents. Please deep-copy the kwargs for each construction and add a regression verifying instance/config isolation.

[willem-bd]

[P2] Remove the guidance-budget regression

This expansion grows the effective middleware guidance chain from 98,459 to 98,565 bytes, above the 98,304-byte limit. The checker permits existing over-budget chains only when they do not grow, so the additional 106 bytes trigger AG002. The exact PR-mode command fails locally, matching the failed GitHub agent-guidance job:

python3 scripts/check_agent_guidance.py --base-ref c9c7076ba74806bd94879cd666b9215bf13c1cb1 --head-ref 08907d9ea1d1d963303a3830249c15bef4d9c484 --github-annotations

Please shorten the guidance enough to eliminate this growth, preferably bringing the chain below the hard limit.