Skip to content

[FLINK-33704][BP 1.18][Filesytems] Update GCS filesystems to latest available versions #23935

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Dec 18, 2023
Merged

[FLINK-33704][BP 1.18][Filesytems] Update GCS filesystems to latest available versions #23935

merged 1 commit into from
Dec 18, 2023

Conversation

MartijnVisser
Copy link
Contributor

@MartijnVisser MartijnVisser commented Dec 15, 2023 •
edited
Loading

Unchanged backport of #23837

Verified locally with mvn clean package -DskipTests -Pfast -Pskip-webui-build -U -T4, changing flink-conf.yaml to include:

state.backend.type: rocksdb
state.checkpoints.dir: gs://mvisser-flink-gcs-test/flink-snapshots
state.backend.incremental: true

And then run a job on this cluster

@MartijnVisser
[FLINK-33704][Filesytems] Update GCS filesystems to latest available …
a6d11e1 
…versions. This closes apache#23837

* [FLINK-33704][Filesystem] Update GCS filesystem dependencies

- Update GS SDK from 2.15.0 to 2.29.1
- Update GS Hadoop Connector from 2.2.15 to 2.2.18
- Update GS GRPC to highest shared version

* [FLINK-33704] Make NOTICE inclusion of com.google.re2j for GCS consistent

RE2J is listed for the other filesystems as Go license, we should remain consistent throughout the project

(cherry picked from commit 492a886)
@MartijnVisser MartijnVisser mentioned this pull request Dec 15, 2023
Closed
@flinkbot
Copy link
Collaborator

flinkbot commented Dec 15, 2023 •
edited
Loading

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

JingGe
JingGe approved these changes Dec 15, 2023
View reviewed changes
Copy link
Contributor

@JingGe JingGe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @MartijnVisser for driving it! LGTM

@snuyanzin
Copy link
Contributor

Unchanged backport of #2383

is this a really correct link?
for me it shows something from 2016...

@czchen
Copy link
Contributor

czchen commented Dec 18, 2023

Shall be #23837

@MartijnVisser
Copy link
Contributor Author

Shall be #23837

Indeed, thanks for that!

@MartijnVisser MartijnVisser merged commit 8951f90 into apache:release-1.18 Dec 18, 2023
@MartijnVisser MartijnVisser deleted the bp-FLINK-33704-to-1.18 branch December 18, 2023 08:05
@ruibinx ruibinx mentioned this pull request Dec 29, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JingGe JingGe JingGe approved these changes

Assignees
No one assigned
Labels
component=Connectors/FileSystem component=FileSystems
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@MartijnVisser @flinkbot @snuyanzin @czchen @JingGe