You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Extend #600 credential refresher to all expiring credential sources (instance roles, Pod Identity) via a CanExpire gate #601
The #600 fix gives IRSA deployments an Arc-managed credential refresher: credentials resolved via the Go AWS SDK, handed to DuckDB as session credentials, re-issued before expiry (internal/database/s3refresh.go). Its detection gate is deliberately narrow — no static keys and both AWS_ROLE_ARN + AWS_WEB_IDENTITY_TOKEN_FILE present — because IRSA is the case we could verify against live AWS STS.
Two credential sources with the same underlying DuckDB expiry bug are NOT covered:
EKS Pod Identity — injects AWS_CONTAINER_CREDENTIALS_FULL_URI / AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE; neither IRSA env var is set, so the gate never fires. DuckDB has no matching chain either, so those deployments have no working path at all today.
Proposal
Broaden the gate from "IRSA env detected" to "the SDK chain resolved credentials with CanExpire=true":
No static keys configured (config or env) → attempt a Go-SDK resolve at startup.
Resolve succeeds with CanExpire=true → start the refresher (covers IRSA, instance roles, Pod Identity, SSO — anything the SDK chain handles).
Resolve succeeds with CanExpire=false → emit once, no loop.
Resolve fails (keyless MinIO / anonymous) → fall back to today's plain CREDENTIAL_CHAIN emission.
The refresher itself already handles all of this (CanExpire branches exist); only the gate in configureS3Access / ConfigureS3 / New changes.
The broadened gate's new cells (IMDS, Pod Identity) could not be live-tested in the time available, and shipping untested credential-path coverage is how #600's class of bug happens. The IRSA cell was verified against real STS end-to-end; the others deserve the same before the gate widens (an IMDS test needs a real EC2 instance; Pod Identity needs an EKS cluster).
Notes for implementation
The startup Warn in configureS3Access (credential_mode=credential_chain → "temporary credentials will NOT auto-refresh") is the operator-facing marker of the uncovered cell; remove it when this ships.
useWebIdentityChain's doc comment names this follow-up.
Release notes 26.09.1 state the limitation publicly ("Not covered yet, stated plainly").
Summary
The #600 fix gives IRSA deployments an Arc-managed credential refresher: credentials resolved via the Go AWS SDK, handed to DuckDB as session credentials, re-issued before expiry (
internal/database/s3refresh.go). Its detection gate is deliberately narrow — no static keys and bothAWS_ROLE_ARN+AWS_WEB_IDENTITY_TOKEN_FILEpresent — because IRSA is the case we could verify against live AWS STS.Two credential sources with the same underlying DuckDB expiry bug are NOT covered:
PROVIDER CREDENTIAL_CHAINresolves them once at CREATE SECRET time. Same failure as IRSA: DuckDB query path fails with ExpiredToken ~1h after start (credential_chain never refreshes) #600, just a longer fuse. Arc currently logs a startup Warn (credential_mode=credential_chain) in this configuration.AWS_CONTAINER_CREDENTIALS_FULL_URI/AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE; neither IRSA env var is set, so the gate never fires. DuckDB has no matching chain either, so those deployments have no working path at all today.Proposal
Broaden the gate from "IRSA env detected" to "the SDK chain resolved credentials with
CanExpire=true":CanExpire=true→ start the refresher (covers IRSA, instance roles, Pod Identity, SSO — anything the SDK chain handles).CanExpire=false→ emit once, no loop.CREDENTIAL_CHAINemission.The refresher itself already handles all of this (
CanExpirebranches exist); only the gate inconfigureS3Access/ConfigureS3/Newchanges.Why it was not shipped in #600
The broadened gate's new cells (IMDS, Pod Identity) could not be live-tested in the time available, and shipping untested credential-path coverage is how #600's class of bug happens. The IRSA cell was verified against real STS end-to-end; the others deserve the same before the gate widens (an IMDS test needs a real EC2 instance; Pod Identity needs an EKS cluster).
Notes for implementation
configureS3Access(credential_mode=credential_chain→ "temporary credentials will NOT auto-refresh") is the operator-facing marker of the uncovered cell; remove it when this ships.useWebIdentityChain's doc comment names this follow-up.