Skip to content

feat(catalog): avoid static global credentials provider #2684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 17, 2025

Conversation

Gezi-lzq
Copy link
Contributor

@Gezi-lzq Gezi-lzq commented Jul 17, 2025

Refactors CredentialProviderHolder to prevent "Connection Pool Shutdown" errors by creating a new provider instance upon each invocation of create().

Previously, a single static AwsCredentialsProvider was shared globally. If this provider was closed, it would affect all subsequent operations. By creating a new provider on each create() call from Iceberg, this change removes the global singleton and isolates provider instances.

Fixes #2680

apache/iceberg#8677

How do I fix "java.lang.IllegalStateException: Connection pool shut down" error?

You attempted to use the credentials provider returned from DefaultCredentialsProvider#create() after it was closed. DefaultCredentialsProvider#create returns a singleton instance, so if it's closed and your code calls the resolveCredentials method, the exception is thrown after cached credentials (or token) expire.

https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/troubleshooting.html#faq-connection-pool-shutdown-exception

Refactors `CredentialProviderHolder` to prevent "Connection Pool Shutdown"
errors by creating a new provider instance for each catalog.

Previously, a single static `AwsCredentialsProvider` was shared globally.
If this provider was closed, it would affect all subsequent operations.
By creating a new provider on each `create()` call from Iceberg, this
change removes the global singleton and isolates provider instances.

Fixes #2680
@Gezi-lzq Gezi-lzq requested review from Copilot and removed request for superhx, SCNieh and Chillax-0v0 July 17, 2025 09:12
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the CredentialProviderHolder class to eliminate the use of a static global credentials provider singleton and instead create a new provider instance on each create() method invocation. This change prevents "Connection Pool Shutdown" errors that occur when a shared static AWS credentials provider is closed, affecting all subsequent operations.

  • Removed static provider field and replaced with bucketURI storage
  • Modified create() method to invoke providerSupplier.apply(bucketURI) for fresh provider instances
  • Updated setup(BucketURI) method to store the bucketURI instead of creating a provider immediately

….java

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Gezi-lzq <lzqtxwd@gmail.com>
@superhx superhx merged commit 5a8aaaa into main Jul 17, 2025
6 checks passed
@superhx superhx deleted the feat/issue-2680-credential-provider branch July 17, 2025 11:19
Gezi-lzq added a commit that referenced this pull request Jul 18, 2025
* feat(catalog): Avoid static global credentials provider

Refactors `CredentialProviderHolder` to prevent "Connection Pool Shutdown"
errors by creating a new provider instance for each catalog.

Previously, a single static `AwsCredentialsProvider` was shared globally.
If this provider was closed, it would affect all subsequent operations.
By creating a new provider on each `create()` call from Iceberg, this
change removes the global singleton and isolates provider instances.

Fixes #2680

* Update core/src/main/java/kafka/automq/table/CredentialProviderHolder.java

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Gezi-lzq <lzqtxwd@gmail.com>

* fix(credentials): update DefaultCredentialsProvider instantiation to use builder pattern

---------

Signed-off-by: Gezi-lzq <lzqtxwd@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
superhx pushed a commit that referenced this pull request Jul 18, 2025
* feat(catalog): Avoid static global credentials provider

Refactors `CredentialProviderHolder` to prevent "Connection Pool Shutdown"
errors by creating a new provider instance for each catalog.

Previously, a single static `AwsCredentialsProvider` was shared globally.
If this provider was closed, it would affect all subsequent operations.
By creating a new provider on each `create()` call from Iceberg, this
change removes the global singleton and isolates provider instances.

Fixes #2680

* Update core/src/main/java/kafka/automq/table/CredentialProviderHolder.java




* fix(credentials): update DefaultCredentialsProvider instantiation to use builder pattern

---------

Signed-off-by: Gezi-lzq <lzqtxwd@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Table Topic - java.lang.IllegalStateException: Connection pool shut down
2 participants