Add workload identity federation via RFC 8693 token exchange

[jjcollinge]

Description

Lets the Dapr clients authenticate with a short-lived token obtained through workload identity federation,
instead of a static DAPR_API_TOKEN. The workload's platform, such as Kubernetes or a cloud provider,
issues it an identity token. The client exchanges that token at an RFC 8693 token exchange endpoint and
sends the result as dapr-api-token. No credential has to be provisioned up front, and the token is
exchanged again ahead of its expiry.

New dapr.credentials package

• CredentialProvider / WorkloadCredential: a provider returns a token with an optional expiry.
• CredentialManager / AsyncCredentialManager: cache the token and refresh it in the background ahead
  of expiry. A failed refresh keeps the last good token and retries. get() never blocks on an in-flight
  refresh.
• oauth2.RFC8693TokenExchangeProvider: exchanges a subject token for an access token, with optional
  audience and scope. RFC 6749 errors are surfaced, and a malformed expires_in is rejected.
• Attestation sources for the subject token:
  • FileAttestationSource: re-reads the file on every exchange, so tokens rotated on disk are picked up.
  • kubernetes_service_account_source(): a file source for a projected service account token.
  • CallableAttestationSource: for metadata endpoints or CI OIDC endpoints.

Client wiring

• DaprClient, DaprGrpcClient, DaprGrpcClientAsync, DaprHttpClient and DaprInvocationHttpClient
  accept an optional credential_manager.
• On gRPC, the token is attached per call through call credentials, so a refreshed token needs no new
  channel.
• On HTTP, it is sent as a header.
• A credential manager takes precedence over DAPR_API_TOKEN and requires a TLS endpoint.

Configuration

• Without code changes, set DAPR_WORKLOAD_IDENTITY_TOKEN_URL and
  DAPR_WORKLOAD_IDENTITY_TOKEN_FILE_PATH, optionally with DAPR_WORKLOAD_IDENTITY_AUDIENCE and
  DAPR_WORKLOAD_IDENTITY_SCOPE.
• Setting only one of the two required settings is an error.
• If DAPR_API_TOKEN is also set, a warning is logged and the workload identity token is used.

Packaging

• New extra dapr[workload-identity] (httpx), included in dapr[all].
• The core install gains no dependencies, and import dapr.clients works without the extra.

Fixes to existing code

• DaprHttpClient.send_bytes wrote into its mutable default headers={}. Auth headers from one call then
  leaked into later calls made without explicit headers, including the actor HTTP client's. It now copies
  the headers.
• create_aio_channel created a plaintext channel, dropping explicit credentials, unless the endpoint had
  ?tls=true. Explicit credentials now always produce a secure channel.

Issue reference

Please reference the issue this PR will close: #[issue number]

Checklist

• Code compiles correctly
• Created/updated tests
• Extended the documentation

[codecov]

Codecov Report

❌ Patch coverage is 97.61905% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.34%. Comparing base (fb229bc) to head (43b1c46).

Files with missing lines	Patch %	Lines
dapr/credentials/manager.py	97.16%	4 Missing ⚠️
dapr/clients/grpc/client.py	89.28%	3 Missing ⚠️
dapr/aio/clients/grpc/client.py	92.85%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1245      +/-   ##
==========================================
+ Coverage   83.89%   84.34%   +0.44%     
==========================================
  Files         123      131       +8     
  Lines       10265    10636     +371     
==========================================
+ Hits         8612     8971     +359     
- Misses       1653     1665      +12     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[CasperGN]

Request changes

The design is sound: bearer tokens go through per-call metadata, mTLS goes through a channel swap with a drain period, and the providers are well separated. Two things need fixing before this leaves draft.

Blocking: the HTTP path sends bearer tokens over plain http://. The PR says a credential manager "requires a TLS endpoint". gRPC enforces that indirectly, because explicit credentials force secure_channel. DaprHttpClient checks nothing. I pointed DaprHttpClient(credential_manager=<RFC8693 manager>) at a local http://127.0.0.1 server and it received dapr-api-token: SECRET-ACCESS-TOKEN in cleartext. With the env-var opt-in and the default DAPR_HTTP_ENDPOINT, every workload-identity user ends up here. Please reject a bearer manager on a non-TLS endpoint, in both the HTTP and gRPC constructors. gRPC currently upgrades a plaintext endpoint to TLS without saying so, and the failure only shows up as a handshake error on the first call.

python-sdk/dapr/clients/http/client.py

Lines 95 to 100 in 9e0c7eb

	if self._credential_manager is not None:
	# Any credential manager replaces DAPR_API_TOKEN; mTLS authenticates in the handshake.
	if self._credential_manager.kind == BEARER:
	headers_map.update(await get_bearer_header(self._credential_manager))
	elif settings.DAPR_API_TOKEN is not None:
	headers_map[DAPR_API_TOKEN_HEADER] = settings.DAPR_API_TOKEN

python-sdk/dapr/clients/grpc/_channel.py

Lines 106 to 107 in 9e0c7eb

	# gRPC only sends call credentials over a secure channel, so explicit credentials imply TLS.
	if uri.tls or credentials is not None:

Blocking: DaprHealth.wait_for_sidecar() ignores the credential manager. Every client constructor calls it first. It sends only DAPR_API_TOKEN and never presents a client certificate. Against an endpoint that requires client certs, with DAPR_CLIENT_CERT_CHAIN_PATH/_PRIVATE_KEY_PATH set, the check fails with TLSV13_ALERT_CERTIFICATE_REQUIRED until it times out, so the client can never be constructed. It also still sends the static token that the PR says the credential manager replaces. The same applies to dapr/aio/clients/health.py. Every end-to-end test patches wait_for_sidecar to a no-op, so none of them catch this.

python-sdk/dapr/clients/health.py

Lines 37 to 48 in 9e0c7eb

	def wait_for_sidecar():
	health_url = f'{get_api_url()}/healthz/outbound'
	headers = {USER_AGENT_HEADER: DAPR_USER_AGENT}
	if settings.DAPR_API_TOKEN is not None:
	headers[DAPR_API_TOKEN_HEADER] = settings.DAPR_API_TOKEN
	timeout = float(settings.DAPR_HEALTH_TIMEOUT)
	start = time.time()
	while True:
	try:
	req = urllib.request.Request(health_url, headers=headers)
	with urllib.request.urlopen(req, context=DaprHealth.get_ssl_context()) as response:

Non-blocking: the workflow extension doesn't use any of this. DaprWorkflowClient and WorkflowRuntime still read DAPR_API_TOKEN directly. Anyone who sets the env vars and uses workflows gets the old auth with no warning. Either wire it in or document the scope clearly.

python-sdk/dapr/ext/workflow/dapr_workflow_client.py

Lines 81 to 82 in 9e0c7eb

	if settings.DAPR_API_TOKEN:
	metadata = ((DAPR_API_TOKEN_HEADER, settings.DAPR_API_TOKEN),)

Non-blocking: the async client now fails outside a running loop when the env vars are set. dapr.aio.clients.DaprClient() built at module level raises a bare RuntimeError: no running event loop once DAPR_CLIENT_* is set. The same code works without them. A clear error message would help, or defer ensure_current_loop and the start() task to first use.

python-sdk/dapr/aio/clients/grpc/client.py

Lines 166 to 173 in 9e0c7eb

	if self._credential_manager is not None:
	self._credential_manager.ensure_current_loop()
	if self._credential_manager.kind == MTLS:
	# The channel is built before __init__ returns; bearer tokens are fetched per
	# call instead, so only mTLS blocks here.
	self._credential_manager.prime_sync()
	self._init_with_credential_manager(address, interceptors, max_grpc_message_length)
	self._background_start_task = asyncio.ensure_future(self._credential_manager.start())

Non-blocking: secrets show up in repr. WorkloadCredential is a plain dataclass, so repr() prints the private key PEM and the bearer token. Any logger.debug("%r", cred) or traceback local-variable dump leaks them. field(repr=False) on token and private_key is cheap.

python-sdk/dapr/credentials/base.py

Lines 42 to 47 in 9e0c7eb

	kind: CredentialKind
	token: Optional[str] = None
	certificate_chain: Optional[bytes] = None
	private_key: Optional[bytes] = None
	trust_bundle: Optional[bytes] = None
	expires_at: Optional[datetime] = None

Non-blocking: test gaps found by mutation. Three mutations survived: dropping suppress_api_token_interceptor from the aio channel (only the sync client has a "replaces DAPR_API_TOKEN" test), ignoring refresh_margin, and removing the per-loop lock rebuild in AsyncCredentialManager._current_lock. The last one passes test_get_is_reusable_across_independent_event_loops anyway, so either the rebuild is dead code or the test doesn't exercise it.

Docs: this adds new env vars, extras and a constructor parameter, so it needs a dapr/docs PR. The example README alone isn't enough.

What I checked: all three commits at 9e0c7eb. tests/credentials: 119 passed. Full unit suite: 1819 passed. ruff check, ruff format --check and mypy are clean. I ran 12 mutations against tests/credentials + tests/clients: 9 killed, including the send_bytes mutable-default fix, drain grace, listener notify, retry-after-failure and kind validation; 3 survived, listed above. I also wrote local repros for the plaintext-HTTP token leak, the health check failing under client-cert mTLS, and the aio client failing outside a loop.