feat(openfeature): harden provider lifecycle and delivery recovery

[pavlokhrebto]

What does this PR do?

Hardens Datadog OpenFeature provider and configuration-delivery lifecycles. It preserves Remote Configuration state across provider replacement, restarts Remote Configuration and agentless delivery after forks, emits ready, changed, and stale provider events, and synchronizes runtime Remote Configuration registration.

It also pins the system-tests workflow to the agentless configuration-source coverage introduced by DataDog/system-tests#7683.

Motivation:

This hardening was extracted from #6295 into a fifth focused PR to keep each layer reviewable. The complete feature is intentionally split across five stacked PRs and will be merged atomically, starting from the last PR.

Change log entry

None. The customer-facing agentless configuration-delivery entry is included in #6295.

Additional Notes:

Warning

This is PR 5 of a 5-PR stack. It depends on code from all four parent PRs and must not be merged directly into master.

Merge the stack last-to-first: #6323 into #6295, #6295 into #6294, #6294 into #6292, #6292 into #6291, and finally #6291 into master.

The split exists only to allow focused reviews; the five PRs form one atomic feature.

How to test the change?

StandardRB, full Steep type checking, and actionlint pass. The OpenFeature rake suite passes on Ruby 3.1 and Ruby 4.0. Agentless configuration-source coverage is provided by DataDog/system-tests#7683.

[linear-code]

FFLSDK-10

[datadog-datadog-prod-us1]

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 96.46%
• Overall Coverage: 90.55% (+0.03%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a1607ad | Docs | View more details | Give us feedback!}

[dd-octo-sts]

Typing analysis

Note: Ignored files are excluded from the next sections.

Untyped methods

This PR introduces 2 partially typed methods, and clears 1 partially typed method. It increases the percentage of typed methods from 72.05% to 72.09% (+0.04%).

Partially typed methods (+2-1)

❌ Introduced:

sig/datadog/core/remote/component.rbs:44
└── def self.build: (
          untyped settings,
          Datadog::Core::Configuration::AgentSettings agent_settings,
          logger: Core::Logger,
          telemetry: Datadog::Core::Telemetry::Component
        ) -> Datadog::Core::Remote::Component?
sig/datadog/open_feature/provider.rbs:94
└── def provider_ready: (Hash[Symbol, untyped] details) -> void

✅ Cleared:

sig/datadog/core/remote/component.rbs:43
└── def self.build: (
          untyped settings,
          Datadog::Core::Configuration::AgentSettings agent_settings,
          logger: Core::Logger,
          telemetry: Datadog::Core::Telemetry::Component,
          ?open_feature_component_provider: (^() -> Datadog::OpenFeature::Component?)?
        ) -> Datadog::Core::Remote::Component?

Untyped other declarations

This PR introduces 1 partially typed other declaration, and clears 1 partially typed other declaration. It increases the percentage of typed other declarations from 86.23% to 86.29% (+0.06%).

Partially typed other declarations (+1-1)

❌ Introduced:

sig/datadog/open_feature/provider.rbs:10
└── type provider_event_handler = ^(Hash[Symbol, untyped]) -> void

✅ Cleared:

sig/datadog/open_feature/provider.rbs:20
└── @error_handler: (^(Hash[Symbol, untyped]) -> void)?

If you believe a method or an attribute is rightfully untyped or partially typed, you can add # untyped:accept on the line before the definition to remove it from the stats.

[pr-commenter]

Benchmarks

Benchmark execution time: 2026-09-28 12:27:36

Comparing candidate commit 3c2db4f in PR branch pavlo.khrebto/FFLSDK-10/provider-lifecycle-hardening with baseline commit 75ba102 in branch pavlo.khrebto/FFLSDK-10/provider-initialization.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 52 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ef1bbff293

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[leoromanovsky]

• init runs asynchronously, but shutdown doesn’t set a terminal/cancelled state. If provider A is replaced by B while A is still waiting for configuration, A can later call activation and overwrite B as the delivery target. A may then emit READY after shutdown, while B stops receiving lifecycle events. See provider.rb and activation.rb. Add a shutdown tombstone/cancellation check before activation and after waiting, plus coverage for shutdown-before-activation and shutdown-while-waiting.

This contradicts the OpenFeature shutdown requirement to end in NOT_READY. The Go provider guards against post-shutdown initialization; Java tears down its evaluator. The new STALE → READY recovery behavior otherwise aligns with Go’s lifecycle semantics.

[pavlokhrebto]

• init runs asynchronously, but shutdown doesn’t set a terminal/cancelled state. If provider A is replaced by B while A is still waiting for configuration, A can later call activation and overwrite B as the delivery target. A may then emit READY after shutdown, while B stops receiving lifecycle events. See provider.rb and activation.rb. Add a shutdown tombstone/cancellation check before activation and after waiting, plus coverage for shutdown-before-activation and shutdown-while-waiting.

This contradicts the OpenFeature shutdown requirement to end in NOT_READY. The Go provider guards against post-shutdown initialization; Java tears down its evaluator. The new STALE → READY recovery behavior otherwise aligns with Go’s lifecycle semantics.

@leoromanovsky thank you for pointing this.

Fixed. Provider shutdown now sets a terminal tombstone and coordinates with activation under the components lock.
Initialization checks cancellation before activation and after waiting, preventing a replaced provider from reclaiming delivery or emitting provider lifecycle events after shutdown.

Added deterministic coverage for shutdown-before-activation and shutdown-while-waiting.

[TonyCTHsu]

Module naming inconsistency: Datadog::OpenFeature vs c.feature_flags

Settings now live under c.feature_flags.*, but the implementation namespace is still Datadog::OpenFeature (lib/datadog/open_feature/). Two vocabularies for one subsystem — the stale settings.open_feature.initialization_timeout_ms references that broke 6292 after the rebase are the kind of bug that split invites.

To be explicit about scope: this means moving the whole module to Datadog::FeatureFlags ("lib/datadog/feature_flags/" — Component, Remote, Activation, evaluators, and the rest), not just the provider constant. Every Datadog::OpenFeature::* constant other than the provider is internal and moves freely; only the two names below need compat shims.

Suggested target for the provider: Datadog::FeatureFlags::OpenFeatureProvider — flat, matching the Vendor::OpenFeatureProvider convention used across the OpenFeature ecosystem (JS LaunchDarklyProvider, Java com.launchdarkly.openfeature.LaunchDarklyProvider), and it pairs the code namespace with the feature_flags settings root. Two compat shims, since the current names are shipped API (Oct 2025):

1. Datadog::OpenFeature::Provider = Datadog::FeatureFlags::OpenFeatureProvider (referenced in customer initializers)
2. keep datadog/open_feature/provider.rb as a require-path shim

If the module name is fixed by the cross-SDK spec, disregard — question raised on #6291.

[Strech]

This PR lacks OOP design and require some common Ruby adjustments. The signs of missing abstractions are:

1. Use of a, b = mutex do statements, that looks like a copy from Go-lang where the tuple return is a design pattern
2. Mixed responsibilities between Configuration, Events and Provider events everything, boarders quite blurry
3. Deeply nested if/else conditions
4. Enormous amount of state/events definitions inside the Provider class, but unrelated to provider perse

[Strech]

For the testing - some Ruby/RSpec hygiene is required. Most of the violations related to definitions, but there are some misuses or testing approaches to be corrected too

[pavlokhrebto]

@TonyCTHsu Thanks—this is a reasonable consistency improvement. I’d prefer not to fold the full namespace migration into this lifecycle stack: Datadog::OpenFeature::Provider and the datadog/open_feature/provider require path predate these PRs and are already shipped API, while feature_flags names the Datadog product settings and OpenFeature names the integration surface. The stale settings reference was fixed and covered as a configuration call-site issue. I suggest handling a canonical Datadog::FeatureFlags::OpenFeatureProvider separately, with compatibility aliases, require-path shims, and an explicit deprecation plan.