Prototype browser lifecycle telemetry with optional console diagnostics

docs/lifecycle-diagnostics-experiment.md

@@ -0,0 +1,33 @@
+# Browser lifecycle diagnostic experiment
+
+This branch adds **experimental lifecycle telemetry** to the precomputed browser provider. Telemetry is emitted whether `debugMode` is omitted, false, or true. The disabled-by-default `debugMode` option controls only local console diagnostics. This is not a production telemetry release; the backend experiment currently accepts diagnostics only for explicitly supported staging test traffic.
+
+```js
+new DatadogProvider({
+  clientToken: 'PUBLIC_CLIENT_TOKEN',
+  applicationId: 'APPLICATION_ID',
+  env: 'staging',
+  site: 'datad0g.com',
+  debugMode: true, // Optional console logging; telemetry is emitted without it.
+})
+```
+
+Debug mode prints local lifecycle and evaluation details, including values, to the developer console. Enable it only where that output is appropriate. Independently, the provider sends lifecycle telemetry in a separate JSONL batch on the existing flag-evaluation transport. Those records use `type: 'sdk_diagnostic'`, `schema_version: 1`, and a bounded diagnostic payload; they contain no flag key, value, context, targeting key, arbitrary error message, stack trace or organization ID. Client tokens authenticate the request, not the payload. No RUM/APM installation or initialization is required, and this works with evaluation/exposure/RUM reporting disabled. Lifecycle upload/queue errors are also printed only when debugMode is enabled; telemetry failures never enable console output on their own.
+
+Events are `sdk_init_started`, `configuration_received`, `provider_ready`, `provider_error`, `init_timeout`, and `init_failed`. The last three use fixed codes `CONFIG_FETCH_FAILED`, `INIT_TIMEOUT`, and `INIT_FAILED`. The fetch code is intentionally generic. There is no `first_evaluation` diagnostic: use genuine evaluation reporting for that stage.
+
+Each provider instance has one runtime ID and emits each transition once (including repeated failures of the same class). A 30-second timer observes slow initialization but does not cancel it; success may arrive later. A failed fetch with cached fallback emits an error without claiming fresh readiness. Superseded requests are not failures. Telemetry upload errors never change evaluation results or generate further diagnostic events. Missing telemetry is unknown, not a confirmed failure.
+
+Remote diagnostics require an `env` plus an `applicationId` or `service`. The payload contains exactly one identity: application_id wins when both are configured; otherwise service_id is the configured service name. Invalid or missing identity/environment leaves diagnostic output local without affecting flag evaluation.
+
+The backend experiment stores every accepted diagnostic in a separate, non-billable storage scope on the evaluation track, configured for one-day retention. It does not sample, aggregate by minute, or deduplicate different records or runtimes. Query results preserve each record's `runtime_id`; they are observed evidence, not proof of current health or of which snippet a person ran. Evaluation queries exclude diagnostics, and evaluation retention is unchanged. The SDK emits once per transition per runtime regardless of debugMode. Retention limits stored history, not the cost of receiving and indexing each event; deployment and effective retention still need staging verification.
+
+Each diagnostic record is limited to 2 KiB and a runtime produces at most six. The existing Browser SDK batching/retry/page-exit helpers supply transport; periodic flushing can delay visibility by up to the normal batch interval (30 seconds), and page-exit delivery is best effort. Closing the provider flushes pending diagnostics and disconnects its page-exit subscription. No new Browser SDK version is needed by this branch.
+
+## Local harness
+
+Run the repository's browser-package installation smoke test (`yarn test:browser-install`) to package the local core/browser SDK and install the test-app dependencies. Then run `yarn dev` from `test-app` and open `/diagnostics.html`.
+
+Enter your staging client token/application/environment. Choose normal, rejected, or delayed configuration fetch; the override does not intercept telemetry upload. Open developer tools before initialization. A clean failure requires no cached/initial configuration: clear this page's IndexedDB or use a fresh profile. Use the close button to flush immediately. Use a separate authenticated management-plane page to query remote evidence—never an API key in this browser app.
+
+The harness optionally reports real evaluations for regression comparisons. Initialization telemetry does not require entering or evaluating a flag. Credentials are not persisted by the harness. This does not replace real staging storage/query verification or establish production retention and cost.

packages/browser/src/domain/configuration.ts

@@ -9,6 +9,12 @@ import { createFlagsConfigurationFetcher } from '../transport/fetchConfiguration
  * Init Configuration for the Flagging SDK.
  */
 export interface FlaggingInitConfiguration extends InitConfiguration {
+  /**
+   * Enable local diagnostic output (default: false). Lifecycle telemetry is emitted independently of this option.
+   * Local evaluation output may contain sensitive flag values. Use temporarily while troubleshooting.
+   */
+  debugMode?: boolean
+
   /**
    * The RUM application ID.
    */

packages/browser/src/openfeature/lifecycleTelemetry.ts

@@ -0,0 +1,121 @@
+import {
+  createBatch,
+  createFlushController,
+  createHttpRequest,
+  createIdentityEncoder,
+  createPageMayExitObservable,
+  generateUUID,
+  Observable,
+  type PageMayExitEvent,
+} from '@datadog/browser-core'
+import type { FlaggingConfiguration } from '../domain/configuration'
+
+type ProgressEvent = 'sdk_init_started' | 'configuration_received' | 'provider_ready'
+type FailureEvent = 'provider_error' | 'init_timeout' | 'init_failed'
+const ERROR_CODES = {
+  provider_error: 'CONFIG_FETCH_FAILED',
+  init_timeout: 'INIT_TIMEOUT',
+  init_failed: 'INIT_FAILED',
+} as const
+
+/** Local diagnostics must never change provider behavior, even with a replaced console. */
+export function logDiagnostic(message: string, details?: unknown): void {
+  try {
+    console.log('[Datadog Feature Flags]', message, details === undefined ? '' : JSON.stringify(details))
+  } catch {
+    // Diagnostic output is best effort and never uploaded.
+  }
+}
+
+/** A separate, bounded lifecycle batch; it does not require evaluation tracking or RUM initialization. */
+export function createLifecycleTelemetry(configuration: FlaggingConfiguration, debugMode = false) {
+  function log(message: string, details?: unknown): void {
+    if (debugMode) logDiagnostic(message, details)
+  }
+
+  const runtimeId = generateUUID()
+  const sent = new Set<ProgressEvent | FailureEvent>()
+  const flushOnClose = new Observable<void>()
+  const pageExit = new Observable<PageMayExitEvent>()
+  const pageSubscription = createPageMayExitObservable(configuration).subscribe((event) => pageExit.notify(event))
+  const batch = createBatch({
+    encoder: createIdentityEncoder(),
+    request: createHttpRequest([configuration.flagEvaluationEndpointBuilder], () => {
+      log('Lifecycle upload failed. Check the client token, site, network, and Content Security Policy.')
+    }),
+    flushController: createFlushController({
+      pageMayExitObservable: pageExit,
+      sessionExpireObservable: flushOnClose,
+    }),
+  })
+  let stopped = false
+
+  function emit(eventType: ProgressEvent | FailureEvent) {
+    if (stopped || sent.has(eventType)) return
+    try {
+      const errorCode = eventType in ERROR_CODES ? ERROR_CODES[eventType as FailureEvent] : undefined
+      log(eventType, errorCode ? { error_code: errorCode } : undefined)
+      // Applications and services have separate remote identities. A browser app wins
+      // when service is also configured for other SDK features.
+      const identity = bounded(configuration.applicationId, 128)
+        ? { application_id: configuration.applicationId }
+        : bounded(configuration.service, 200)
+          ? { service_id: configuration.service }
+          : undefined
+      if (!identity || !bounded(configuration.env, 200)) {
+        sent.add(eventType)
+        log(
+          'Remote lifecycle diagnostics require an applicationId or service and an env. Local diagnostics remain available.'
+        )
+        return
+      }
+      const event = {
+        schema_version: 1,
+        type: 'sdk_diagnostic',
+        payload: {
+          event_type: eventType,
+          timestamp: Date.now(),
+          runtime_id: runtimeId,
+          sdk_name: 'dd-openfeature-browser',
+          sdk_version: __BUILD_ENV__SDK_VERSION__,
+          ...identity,
+          environment: configuration.env,
+          ...(errorCode && { error_code: errorCode }),
+        },
+      }
+      // Bound UTF-8 bytes independently of the transport's much larger generic message limit.
+      if (new Blob([JSON.stringify(event)]).size > 2048) {
+        log('Lifecycle event exceeds the diagnostic size limit.')
+        return
+      }
+      sent.add(eventType)
+      batch.add(event)
+    } catch {
+      log('Unable to queue lifecycle diagnostics. Flag evaluation is unaffected.')
+    }
+  }
+
+  return {
+    emit,
+    stop() {
+      if (stopped) return
+      stopped = true
+      try {
+        if (batch.flushController.messagesCount > 0) flushOnClose.notify()
+      } finally {
+        batch.stop()
+        pageSubscription.unsubscribe()
+      }
+    },
+  }
+}
+
+function bounded(value: string | undefined, limit: number): value is string {
+  return (
+    typeof value === 'string' &&
+    value.length > 0 &&
+    value.length <= limit &&
+    value.trim() === value &&
+    !/\p{Cc}/u.test(value)
+  )
+}

packages/browser/src/openfeature/provider.ts

@@ -26,6 +26,7 @@ import { DatadogCoreProvider } from './core-provider'
 import { toProviderErrorEvent } from './error-event'
 import { createExposureLoggingHook } from './exposures'
 import { createFlagEvalEVPHook } from './flagEvaluations'
+import { createLifecycleTelemetry, logDiagnostic } from './lifecycleTelemetry'
 import { createRumTrackingHook, enrichEvaluationContextWithRumUser } from './rumIntegration'
 
 /**
@@ -64,6 +65,11 @@ export class DatadogProvider extends DatadogCoreProvider {
 
   /** Controls both directions of the provider's RUM integration. */
   private readonly isRumIntegrationEnabled: boolean
+  private readonly debugMode: boolean
+  private lifecycleTelemetry?: ReturnType<typeof createLifecycleTelemetry>
+  private initializationTimer?: ReturnType<typeof setTimeout>
+  private initializationStarted = false
+  private initializationFinished = false
 
   // TODO: Migrate this manual context plumbing to a provider `before` hook once
   // @openfeature/web-sdk supports returned EvaluationContext values for web hooks.
@@ -103,6 +109,7 @@ export class DatadogProvider extends DatadogCoreProvider {
 
   constructor(options: FlaggingInitConfiguration) {
     super()
+    this.debugMode = options.debugMode === true
     this.configuration = validateAndBuildFlaggingConfiguration(options)
 
     // Set up provider-managed hooks and events
@@ -138,8 +145,36 @@ export class DatadogProvider extends DatadogCoreProvider {
   }
 
   async initialize(context: EvaluationContext = {}): Promise<void> {
+    if (!this.initializationStarted) {
+      this.initializationStarted = true
+      if (this.debugMode) {
+        logDiagnostic(
+          'Initializing. If no remote events arrive, check the client token, site, network, and Content Security Policy.'
+        )
+      }
+      try {
+        if (this.configuration) {
+          this.lifecycleTelemetry = createLifecycleTelemetry(this.configuration, this.debugMode)
+          this.lifecycleTelemetry.emit('sdk_init_started')
+          // Observe slow startup without aborting or changing OpenFeature initialization.
+          this.initializationTimer = setTimeout(() => this.lifecycleTelemetry?.emit('init_timeout'), 30_000)
+        }
+      } catch {
+        if (this.debugMode) logDiagnostic('Lifecycle transport unavailable. Inspect local diagnostics instead.')
+      }
+    }
     this.exposureCacheReady = this.exposureCache?.init()
-    return this.setContext(context)
+    try {
+      return await this.setContext(context)
+    } catch (error) {
+      this.lifecycleTelemetry?.emit('init_failed')
+      if (this.debugMode)
+        logDiagnostic('Initialization failed. Inspect the configuration request in browser developer tools.')
+      throw error
+    } finally {
+      this.initializationFinished = true
+      clearTimeout(this.initializationTimer)
+    }
   }
 
   public onContextChange(_oldContext: EvaluationContext, context: EvaluationContext): Promise<void> {
@@ -198,6 +233,9 @@ export class DatadogProvider extends DatadogCoreProvider {
           this.flagsConfiguration = config
           this.evaluationContext = evaluationContext
           this.status = fromCache ? ProviderStatus.STALE : ProviderStatus.READY
+          if (!fromCache && config.precomputed) {
+            this.lifecycleTelemetry?.emit('provider_ready')
+          }
           this.events.emit(ProviderEvents.ConfigurationChanged)
 
           if (this.status === ProviderStatus.STALE) {
@@ -250,9 +288,22 @@ export class DatadogProvider extends DatadogCoreProvider {
 
     try {
       const config = await this.configuration.fetchFlagsConfiguration(context, { signal })
+      if (!signal.aborted) {
+        if (config.precomputed) {
+          this.lifecycleTelemetry?.emit('configuration_received')
+        } else if (config.precomputedError) {
+          this.lifecycleTelemetry?.emit('provider_error')
+          if (!this.initializationFinished) this.lifecycleTelemetry?.emit('init_failed')
+          if (this.debugMode) logDiagnostic('Configuration could not be parsed. Inspect the local response.')
+        }
+      }
       this.flagsCache?.set(config, context)
       return { config, fromCache: false }
     } catch (err) {
+      if (!signal.aborted) {
+        this.lifecycleTelemetry?.emit('provider_error')
+        if (this.debugMode) logDiagnostic('Configuration fetch failed. Cached configuration may still be available.')
+      }
       // Try to recover with current/cached config
       try {
         const config = await waitWithAbort(signal, cachedConfigPromise)
@@ -285,12 +336,23 @@ export class DatadogProvider extends DatadogCoreProvider {
     _context: EvaluationContext,
     _logger: Logger
   ): ResolutionDetails<FlagTypeToValue<T>> {
-    return evaluatePrecomputedConfiguration(
+    const details = evaluatePrecomputedConfiguration(
       this.flagsConfiguration,
       type,
       flagKey,
       defaultValue,
       this.evaluationContext
     )
+    if (this.debugMode) logDiagnostic('Evaluation details', { flagKey, ...details })
+    return details
+  }
+
+  async onClose(): Promise<void> {
+    clearTimeout(this.initializationTimer)
+    try {
+      this.lifecycleTelemetry?.stop()
+    } catch {
+      // Diagnostic shutdown must not interrupt application shutdown.
+    }
   }
 }