feat(openfeature): add agentless EVP fallback

[leoromanovsky]

Motivation

FFL-1485

Go agentless Feature Flags must deliver exposure and flagevaluation events when the local Agent EVP proxy is unavailable.

flowchart LR
  SDK["Go agentless Feature Flags"] --> Signals["Exposures + flag evaluations"]
  Signals --> Receiver["Local Agent or sidecar"]
  Receiver -->|"Compatible EVP route"| Intake["Event Platform intake"]
  Receiver -.->|"Unavailable or unsupported"| Gap["No delivery path"]

Loading

Changes

flowchart LR
  FirstSend["First event send"] --> Discover["Lazy /info discovery via sync.Once"]
  Discover -->|"v4 advertised"| V4["/evp_proxy/v4"]
  Discover -->|"v2 only"| V2["/evp_proxy/v2"]
  Discover -->|"Failure or no compatible route"| Direct["Direct intake + DD-API-KEY"]
  V4 --> Intake["Event Platform intake"]
  V2 --> Intake
  Direct --> Intake

Loading

• Use one EVP client for /api/v2/exposures and /api/v2/flagevaluation.
• Discover Agent EVP support lazily on the first event send and cache the selected route.
• Prefer /evp_proxy/v4, then /evp_proxy/v2, then authenticated direct intake.
• Validate DD_SITE as bounded ASCII DNS labels before constructing the direct intake host.
• Refuse direct-intake redirects before they can forward DD-API-KEY.
• Document the Agent address settings used for Agentless EVP discovery and delivery.
• Add proxy-aware direct HTTPS delivery and focused route, header, site-validation, redirect, failure, and concurrency coverage.
• Update OpenFeature Go SDK to v1.17.2 for non-blocking provider event-handler replacement.

Decisions

• Agent requests send X-Datadog-EVP-Subdomain; direct requests send DD-API-KEY.
• Direct intake uses https://event-platform-intake.<site>:443.
• Direct-intake redirects are returned as errors and never followed.
• HTTP 403/404/405 and definitive pre-send connection failures replay the current batch directly.
• Ambiguous transport failures switch future batches only; HTTP 429 and 5xx remain on the Agent route.
• Direct intake is terminal, and Remote Configuration remains Agent-only.

Validation

Local SDK validation:

• go test ./openfeature
• go test -race ./openfeature
• go test ./openfeature -run '^TestEndToEnd_(ConfigurationUpdate|TrafficSharding)$' -count=100
• GOTOOLCHAIN=go1.26.4 go test ./openfeature -run '^TestEndToEnd_(ConfigurationUpdate|TrafficSharding)$' -count=100
• GOOS=windows GOARCH=amd64 GOTOOLCHAIN=go1.26.4 go test -c ./openfeature
• go mod tidy -diff
• ./bin/golangci-lint run ./openfeature (0 issues)
• gofmt and git diff --check

The Go system tests are defined and enabled in DataDog/system-tests#7604.

Ran the system-test stack against the locally built Go branch artifact in these scenarios:

• FEATURE_FLAGGING_AND_EXPERIMENTATION: exposure and flag-evaluation egress
• FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS_DIRECT: exposure and flag-evaluation egress
• FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS_SERVERLESS: exposure and flag-evaluation egress

In each topology, both egress tests completed successfully. They verified the selected route, direct-intake host, response status and API-key header where applicable, aggregate evaluation counts, and zero events on the unused route.

[linear-code]

FFL-1485

FFL-1482

[datadog-datadog-us1-prod]

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 2 jobs - 2 passed on retry

🎯 Code Coverage (details)
• Patch Coverage: 94.71%
• Overall Coverage: 65.23% (+0.09%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9b9388f | Docs | View more details | Give us feedback!}

[pr-commenter]

Benchmarks

Benchmark execution time: 2026-08-27 08:59:12

Comparing candidate commit 9b9388f in PR branch leo.romanovsky/ffe-agentless-evp-fallback-go-v2 with baseline commit cde3470 in branch pavlo.khrebto/FFL-2695/agentless-events.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 335 metrics, 0 unstable metrics, 1 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

Known flaky benchmarks without significant changes:

• scenario:BenchmarkOTLPTraceWriterFlush

[Copilot]

Pull request overview

This PR adds an agentless EVP delivery fallback for the OpenFeature provider so exposure and flag-evaluation events can still be delivered when the local Agent EVP proxy is unavailable or unsupported. It introduces lazy /info discovery to choose the best local EVP proxy version (v4 → v2) and falls back to direct intake using DD_API_KEY + a validated DD_SITE-derived host, with redirect refusal for credential safety.

Changes:

• Implement agentless EVP route discovery and routing (local v4/v2 → direct intake fallback) with redirect refusal and site validation.
• Switch exposure + flagevaluation endpoints to direct-intake paths and let routing prepend the selected EVP proxy base when using the Agent.
• Add focused tests for route selection, headers, redirects, error/fallback behavior, and concurrency; update docs and bump OpenFeature Go SDK to v1.17.2.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
openfeature/provider.go	Constructs agentless EVP client for agentless configuration source.
openfeature/integration_test.go	Updates end-to-end tests to expect EVP proxy base-path + direct endpoint join behavior.
openfeature/flageval_logging.go	Switches flagevaluation endpoint constant to direct-intake path.
openfeature/flageval_logging_test.go	Updates endpoint expectations and EVP proxy path joining in tests.
openfeature/exposure.go	Switches exposure endpoint constant to direct-intake path.
openfeature/evp.go	Adds route discovery, local/direct routing, redirect refusal, site validation, and error-based fallback logic.
openfeature/evp_test.go	Adds unit tests for routing, headers, redirects, failure matrix, and concurrency.
openfeature/doc.go	Documents EVP discovery/fallback behavior and Agent address env vars used for discovery.
go.mod	Bumps github.com/open-feature/go-sdk to v1.17.2.
go.sum	Updates sums for OpenFeature Go SDK bump.
go.work.sum	Updates workspace sums (includes an additional go.mod checksum entry).

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.