[SPARK-53927][BUILD][DSTREAM][4.0] Upgrade kinesis client and fix kinesis integration tests

[vrozov]

What changes were proposed in this pull request?

Backport of #52630 to 4.0

Why are the changes needed?

branch-4.0 is subject to the same error as master. The integration with kinesis does not work due to dependencies conflict.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Using AWS credentials and ENABLE_KINESIS_TESTS=1 build/mvn test -Pkinesis-asl -pl connector/kinesis-asl

Was this patch authored or co-authored using generative AI tooling?

No

[vrozov]

@sarutak Please review.

[sarutak]

@vrozov

WithAggregationKinesisBackedBlockRDDSuite doesn't seem pass.

[info] WithAggregationKinesisBackedBlockRDDSuite:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "kpl-daemon-0000" java.lang.NoSuchFieldError: AWS_ACCOUNT_ID
	at software.amazon.awssdk.auth.credentials.internal.SystemSettingsCredentialsProvider.resolveCredentials(SystemSettingsCredentialsProvider.java:52)
	at software.amazon.awssdk.auth.credentials.AwsCredentialsProvider.resolveIdentity(AwsCredentialsProvider.java:54)
	at software.amazon.awssdk.identity.spi.IdentityProvider.resolveIdentity(IdentityProvider.java:60)
	at software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:109)
	at software.amazon.awssdk.auth.credentials.internal.LazyAwsCredentialsProvider.resolveCredentials(LazyAwsCredentialsProvider.java:45)
	at software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider.resolveCredentials(DefaultCredentialsProvider.java:129)
	at software.amazon.kinesis.producer.Daemon.makeSetCredentialsMessage(Daemon.java:563)
	at software.amazon.kinesis.producer.Daemon.startChildProcess(Daemon.java:432)
	at software.amazon.kinesis.producer.Daemon.access$100(Daemon.java:60)
	at software.amazon.kinesis.producer.Daemon$1.run(Daemon.java:129)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

Also, please add the target branch to the title like [4.0] for backporting PRs next time.