Skip to content

feat: forward selected webhook headers to DAG runs - #2050

Merged
yohamta0 merged 6 commits into
mainfrom
webhook-forward-headers
Apr 28, 2026
Merged

yohamta0 merged 6 commits into
mainfrom
webhook-forward-headers

Conversation

@yohamta0

@yohamta0 yohamta0 commented Apr 28, 2026 •

Copy link
Copy Markdown
Member

Closes #2037

Summary

  • add DAG-level webhook.forward_headers support and expose forwarded headers as WEBHOOK_HEADERS
  • capture webhook request headers in middleware, filter them against the DAG allowlist, and keep Authorization blocked
  • tighten the implementation by centralizing header policy, extracting webhook runtime-param assembly, and splitting helper tests from end-to-end webhook tests

Why

  • webhook-triggered DAGs need access to selected source headers without forwarding everything
  • keeping the allowlist in DAG YAML preserves a small surface area and avoids a heavier webhook-specific config path
  • the cleanup keeps the new behavior maintainable as webhook support grows

Validation

  • go test ./internal/core ./internal/core/spec ./internal/service/frontend/api/v1 -count=1
  • pnpm --dir ui typecheck

Summary by CodeRabbit

  • New Features

    • Webhook-triggered DAGs can forward a configured subset of incoming HTTP request headers to runs via WEBHOOK_HEADERS; header names are normalized to lowercase and Authorization is never forwarded.
    • Child DAGs may inherit, override, or clear an inherited webhook.forward_headers list.
  • Documentation

    • API docs and UI guidance updated to describe header-forwarding configuration and behavior.

@coderabbitai

coderabbitai Bot commented Apr 28, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 367c9e62-5401-44d4-9448-8acd2849b646

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds per‑DAG webhook.forward_headers allowlist, captures and normalizes request headers in middleware, validates/filters them (never forwarding Authorization), and includes selected headers as a quoted JSON WEBHOOK_HEADERS runtime parameter alongside WEBHOOK_PAYLOAD when enqueuing webhook-triggered DAG runs.

Changes

Cohort / File(s) Summary
Schema & Core Types
internal/cmn/schema/dag.schema.json, internal/core/dag.go
Add top-level webhook schema and core.WebhookConfig with forward_headers; attach Webhook *WebhookConfig to DAG and deep-copy slice in DAG.Clone().
Spec Builders & Validation
internal/core/spec/dag.go, internal/core/spec/dag_test.go, internal/core/spec/loader.go, internal/core/spec/loader_test.go, internal/core/spec/loader_internal_test.go
Introduce webhook transformer that normalizes/validates headers (reject empty/invalid tokens and authorization), add merge rule to replace webhook config, and tests for build/merge/decoder hints.
Core Webhook Utilities
internal/core/webhook.go, internal/core/webhook_test.go
New helpers to normalize header names, validate token grammar (RFC-like), and identify denied headers (authorization); unit tests added.
Runtime Params & Key Hints
internal/core/spec/dag_params_runtime.go, internal/core/spec/runtime_params_test.go, internal/core/spec/key_hints.go
Mark WEBHOOK_HEADERS as an internal runtime param; add tests for runtime param resolution and legacy key hint mapping.
Middleware & Request Context
internal/service/frontend/api/v1/webhook_middleware.go, internal/service/frontend/api/v1/api.go
Rename middleware to WebhookRequestContextMiddleware; middleware now stores raw body and a cloned/normalized header map in request context; update route registration.
Webhook Handler & Helpers
internal/service/frontend/api/v1/webhooks.go, internal/service/frontend/api/v1/webhook_helpers_test.go, internal/service/frontend/api/v1/export_test.go
TriggerWebhook now builds runtime params via helpers to include WEBHOOK_PAYLOAD and filtered WEBHOOK_HEADERS (never forwards Authorization); add helper tests and export test hooks.
Handler Tests / Integration
internal/service/frontend/api/v1/webhooks_test.go
Replace focused unit tests with an end‑to‑end test validating header forwarding semantics, exclusion of authorization, and OS-specific capture step differences.
API Spec & UI Docs
api/v1/api.yaml, api/v1/api.gen.go, ui/src/api/v1/schema.ts, ui/src/features/dags/components/dag-details/WebhookTab.tsx
Document optional WEBHOOK_HEADERS forwarding and exclusion of Authorization in OpenAPI, regenerated embedded spec, and update UI docs/examples.
File Utilities
internal/cmn/fileutil/fileutil.go, internal/cmn/fileutil/fileutil_test.go
Sanitize dagName when creating temp DAG files (strip path and extensions) and add test for basename handling.

Sequence Diagram(s)

sequenceDiagram
  participant Client as Client
  participant Frontend as FrontendAPI
  participant Middleware as Middleware (WebhookRequestContextMiddleware)
  participant Handler as TriggerWebhook Handler
  participant Spec as DAG Spec/Loader
  participant Queue as Queue

  Client->>Frontend: POST /api/v1/webhooks/{id} (body + headers)
  Frontend->>Middleware: pass request
  Middleware-->>Frontend: attach raw body + normalized headers to context
  Frontend->>Handler: call TriggerWebhook with ctx
  Handler->>Spec: load DAG and webhook.forward_headers
  Handler->>Handler: marshal payload -> WEBHOOK_PAYLOAD
  Handler->>Handler: select/filter headers -> marshal -> WEBHOOK_HEADERS
  Handler->>Queue: enqueue DAG run with runtime params
  Queue-->>Client: respond with dag-run ID
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related issues

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 34.15% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat: forward selected webhook headers to DAG runs' directly and clearly describes the main feature being added: the ability to forward selected webhook request headers to DAG runs via configuration.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch webhook-forward-headers

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@api/v1/api.yaml`:
- Around line 3860-3863: The documentation claims selected request headers are
passed as the WEBHOOK_HEADERS environment variable when webhook.forward_headers
is configured, but it omits that the Authorization header is explicitly blocked;
update the sentence describing webhook.forward_headers/WEBHOOK_HEADERS to add a
clear caveat that the Authorization header will never be forwarded (i.e.,
Authorization is always excluded from WEBHOOK_HEADERS).

In `@internal/core/spec/dag.go`:
- Around line 884-901: The webhook forward header validation currently only
checks for empty strings and rejects "Authorization" but accepts invalid HTTP
header token names; update the parsing/validation in the block handling
d.Webhook.ForwardHeaders (where core.NormalizeWebhookForwardHeader and
core.IsDeniedWebhookForwardHeader are used) to additionally validate that the
normalized header matches the RFC9110 token grammar (tchar set: alphanumerics
and !#$%&'*+-.^_`|~) and return a core.NewValidationError (same pattern as
existing errors) when it does not, before appending to headers; implement the
token check either by calling or adding a helper like core.IsValidHeaderToken
and use it in this loop to reject entries such as "my header".

In `@internal/core/spec/key_hints.go`:
- Line 34: Add a unit test that exercises the key hint lookup for the
"forwardHeaders" → "webhook.forward_headers" mapping found in key_hints.go:
create a webhook config using the camelCase key "forwardHeaders" (instead of
"forward_headers"), run the same validation/parse path used by existing webhook
tests, and assert that the returned error message includes the hint recommending
"webhook.forward_headers"; this ensures the mapping for the symbol
"forwardHeaders" in key_hints.go is covered and the suggestion text appears in
the error output.

In `@ui/src/features/dags/components/dag-details/WebhookTab.tsx`:
- Around line 982-992: The copy mistakenly suggests webhook.forward_headers can
be set "in the DAG or base config" while this feature is DAG-scoped; update the
text in WebhookTab (the list item referring to webhook.forward_headers and
WEBHOOK_HEADERS) to remove "or base config" and explicitly state that
webhook.forward_headers is configured in the DAG only (or otherwise reword to
indicate DAG scope) so it doesn't imply a base-config option.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c8ab9137-e26d-44b7-b62b-6296b26c6d3b

📥 Commits

Reviewing files that changed from the base of the PR and between 9d0668c and 39773d9.

📒 Files selected for processing (21)
  • api/v1/api.gen.go
  • api/v1/api.yaml
  • internal/cmn/schema/dag.schema.json
  • internal/core/dag.go
  • internal/core/spec/dag.go
  • internal/core/spec/dag_params_runtime.go
  • internal/core/spec/dag_test.go
  • internal/core/spec/key_hints.go
  • internal/core/spec/loader.go
  • internal/core/spec/loader_test.go
  • internal/core/spec/runtime_params_test.go
  • internal/core/webhook.go
  • internal/core/webhook_test.go
  • internal/service/frontend/api/v1/api.go
  • internal/service/frontend/api/v1/export_test.go
  • internal/service/frontend/api/v1/webhook_helpers_test.go
  • internal/service/frontend/api/v1/webhook_middleware.go
  • internal/service/frontend/api/v1/webhooks.go
  • internal/service/frontend/api/v1/webhooks_test.go
  • ui/src/api/v1/schema.ts
  • ui/src/features/dags/components/dag-details/WebhookTab.tsx

Comment thread api/v1/api.yaml Outdated
Comment thread internal/core/spec/dag.go
Comment thread internal/core/spec/key_hints.go
Comment thread ui/src/features/dags/components/dag-details/WebhookTab.tsx
@yohamta0

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Apr 28, 2026

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@yohamta0

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Apr 28, 2026

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@internal/cmn/fileutil/fileutil.go`:
- Around line 214-221: Trim dagName into a local variable first and treat a
whitespace-only value as empty: after computing trimmed :=
strings.TrimSpace(dagName) set patternName to a safe fallback (e.g., "dag" or
another default) when trimmed == "" otherwise set patternName =
filepath.Base(trimmed) and then remove YAML extension as you already do; finally
use TruncString(patternName, maxTempPrefix) to build pattern — this ensures
patternName never contains only spaces and avoids unsafe prefixes when creating
pattern.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 644c3573-6c55-48e1-b829-80240ae3654e

📥 Commits

Reviewing files that changed from the base of the PR and between 28cc4d4 and 9c935b3.

📒 Files selected for processing (3)
  • internal/cmn/fileutil/fileutil.go
  • internal/cmn/fileutil/fileutil_test.go
  • internal/service/frontend/api/v1/webhooks_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • internal/service/frontend/api/v1/webhooks_test.go

Comment thread internal/cmn/fileutil/fileutil.go Outdated
@yohamta0
yohamta0 merged commit a14fd62 into main Apr 28, 2026
11 checks passed
@yohamta0
yohamta0 deleted the webhook-forward-headers branch April 28, 2026 23:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Forward selected webhook request headers to triggered DAG runs

1 participant