AWS SDK v2 migration #2339
Merged
AWS SDK v2 migration #2339
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Trianz-Akshay <108925344+Trianz-Akshay@users.noreply.github.com> Co-authored-by: Jithendar Trianz <106380520+Jithendar12@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Mario Rial <mr2224@cornell.edu> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: AbdulRehman <ar.al-faraj@outlook.com>
Co-authored-by: Aimery Methena <159072740+aimethed@users.noreply.github.com>
Co-authored-by: Trianz-Akshay <108925344+Trianz-Akshay@users.noreply.github.com>
burhan94
approved these changes
Oct 22, 2024
There was a problem hiding this comment.
So I went through this once. I definitely missed something but its going to be almost impossible to find anything without spending multiple days on this. It seems like all tests are passing and the Trianz folks also ran tests. I'm comfortable with approving since there isn't a better option here
Closed
github-actions bot
pushed a commit
that referenced
this pull request
Oct 23, 2024
- update validation_testing to use ECR (#2351) - Opt in imageuri (#2340) - build(deps): bump software.amazon.glue:schema-registry-serde from 1.1.20 to 1.1.21 (#2349) - build(deps): bump software.amazon.glue:schema-registry-serde - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client from 8.15.2 to 8.15.3 (#2348) - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client - build(deps): bump com.mysql:mysql-connector-j from 9.0.0 to 9.1.0 (#2346) - build(deps): bump com.mysql:mysql-connector-j from 9.0.0 to 9.1.0 - AWS SDK v2 migration (#2339) - build(deps): bump software.amazon.awssdk:bom from 2.28.21 to 2.28.26 (#2343) - build(deps): bump software.amazon.awssdk:bom from 2.28.21 to 2.28.26 - Feature/postgresql disable predicate pushdown (#2337) - Elasticsearch geo_point type support by converting to varchar (#2244) - build(deps): bump org.eclipse.jetty:jetty-server from 11.0.16 to 11.0.24 in /athena-hbase (#2336) - build(deps): bump org.eclipse.jetty:jetty-server in /athena-hbase - build(deps): bump org.jetbrains.kotlin:kotlin-reflect from 2.0.20 to 2.0.21 (#2334) - build(deps): bump org.jetbrains.kotlin:kotlin-reflect - build(deps): bump surefire.failsafe.version from 3.5.0 to 3.5.1 (#2316) - build(deps): bump surefire.failsafe.version from 3.5.0 to 3.5.1 - build(deps): bump software.amazon.awssdk:bom from 2.28.16 to 2.28.21 (#2329) - build(deps): bump software.amazon.awssdk:bom from 2.28.16 to 2.28.21 - build(deps): bump org.jetbrains.kotlin:kotlin-stdlib from 2.0.20 to 2.0.21 (#2335) - build(deps): bump org.jetbrains.kotlin:kotlin-stdlib - build(deps): bump software.amazon.jsii:jsii-runtime from 1.103.1 to 1.104.0 (#2333) - build(deps): bump software.amazon.jsii:jsii-runtime - build(deps): bump com.sap.cloud.db.jdbc:ngdbc from 2.22.11 to 2.22.12 (#2332) - build(deps): bump com.sap.cloud.db.jdbc:ngdbc from 2.22.11 to 2.22.12 - build(deps): bump org.jetbrains.kotlin:kotlin-stdlib-jdk8 from 2.0.20 to 2.0.21 (#2331) - build(deps): bump org.jetbrains.kotlin:kotlin-stdlib-jdk8 - build(deps): bump com.google.cloud:google-cloud-storage from 2.43.1 to 2.43.2 (#2330) - build(deps): bump com.google.cloud:google-cloud-storage - build(deps): bump org.jetbrains.kotlin:kotlin-stdlib-common from 2.0.20 to 2.0.21 (#2328) - build(deps): bump org.jetbrains.kotlin:kotlin-stdlib-common - [Fix] athena-sqlserver: Add null check for partition function and partitioning column in getPartitionWhereClauses method (#2322) - build(deps): bump software.amazon.awssdk:bom from 2.28.11 to 2.28.16 (#2320) - build(deps): bump software.amazon.awssdk:bom from 2.28.11 to 2.28.16 - build(deps): bump net.jqwik:jqwik from 1.9.0 to 1.9.1 (#2321) - build(deps): bump net.jqwik:jqwik from 1.9.0 to 1.9.1 - build(deps): bump org.apache.maven.plugins:maven-javadoc-plugin from 3.10.0 to 3.10.1 (#2315) - build(deps): bump org.apache.maven.plugins:maven-javadoc-plugin - build(deps-dev): bump nl.jqno.equalsverifier:equalsverifier from 3.17 to 3.17.1 (#2318) - build(deps-dev): bump nl.jqno.equalsverifier:equalsverifier - build(deps): bump org.junit:junit-bom from 5.11.1 to 5.11.2 (#2317) - build(deps): bump org.junit:junit-bom from 5.11.1 to 5.11.2 - build(deps): bump aws-sdk.version from 1.12.772 to 1.12.773 (#2314) - build(deps): bump aws-sdk.version from 1.12.772 to 1.12.773 - [Fix] athena-synapse: Add missing s3 permissions to spill the data that exceeds lambda function limits (#2309) - error handling changes for dynamodb (#2277) - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client from 8.15.1 to 8.15.2 (#2303) - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client - build(deps): bump com.microsoft.azure:msal4j from 1.17.1 to 1.17.2 (#2306) - build(deps): bump com.microsoft.azure:msal4j from 1.17.1 to 1.17.2 - build(deps): bump org.junit:junit-bom from 5.11.0 to 5.11.1 (#2304) - build(deps): bump org.junit:junit-bom from 5.11.0 to 5.11.1 - build(deps): bump software.amazon.awssdk:bom from 2.28.6 to 2.28.11 (#2299) - build(deps): bump software.amazon.awssdk:bom from 2.28.6 to 2.28.11 - build(deps): bump org.apache.maven.plugins:maven-gpg-plugin from 3.2.6 to 3.2.7 (#2307) - build(deps): bump org.apache.maven.plugins:maven-gpg-plugin - build(deps): bump fasterxml.jackson.version from 2.17.2 to 2.18.0 (#2302) - build(deps): bump fasterxml.jackson.version from 2.17.2 to 2.18.0 - build(deps-dev): bump log4j2Version from 2.24.0 to 2.24.1 (#2300) - build(deps-dev): bump log4j2Version from 2.24.0 to 2.24.1 - build(deps): bump com.google.guava:guava from 33.3.0-jre to 33.3.1-jre (#2301) - build(deps): bump com.google.guava:guava from 33.3.0-jre to 33.3.1-jre - build(deps): bump com.google.cloud:google-cloud-storage from 2.43.0 to 2.43.1 (#2305) - build(deps): bump com.google.cloud:google-cloud-storage - build(deps-dev): bump nl.jqno.equalsverifier:equalsverifier from 3.16.2 to 3.17 (#2298) - build(deps-dev): bump nl.jqno.equalsverifier:equalsverifier
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
#1960
Description of changes:
Migrate the codebase to use AWS SDK v2 (from v1). Because this increases the jar sizes of connectors, some exceeded the limit allowed by SAM or lambda. To get around this, image based lambdas are used. The connectors are built and pushed to private ECR repositories that are exposed to the public (lambda does not support public ECR) using the Dockerfiles included in this PR.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.