Update all non-major dependencies

[renovate]

This PR contains the following updates:

Package	Change	Age	Adoption	Passing	Confidence
cloud.google.com/go/pubsub	v1.33.0 -> v1.36.1
github.com/apache/beam/sdks/v2	v2.51.0 -> v2.54.0

---

Release Notes

apache/beam (github.com/apache/beam/sdks/v2)

v2.54.0: Beam 2.54.0 release

We are happy to present the new 2.54.0 release of Beam.
This release includes both improvements and new functionality.
See the download page for this release.

For more information on changes in 2.54.0, check out the detailed release notes.

Highlights

• Enrichment Transform along with GCP BigTable handler added to Python SDK (#​30001).
• Beam Java Batch pipelines run on Google Cloud Dataflow will default to the Portable (Runner V2)[https://cloud.google.com/dataflow/docs/runner-v2] starting with this version. (All other languages are already on Runner V2.)
  • This change is still rolling out to the Dataflow service, see (Runner V2 documentation)[https://cloud.google.com/dataflow/docs/runner-v2] for how to enable or disable it intentionally.

I/Os

• Added support for writing to BigQuery dynamic destinations with Python's Storage Write API (#​30045)
• Adding support for Tuples DataType in ClickHouse (Java) (#​29715).
• Added support for handling bad records to FileIO, TextIO, AvroIO (#​29670).
• Added support for handling bad records to BigtableIO (#​29885).

New Features / Improvements

• Enrichment Transform along with GCP BigTable handler added to Python SDK (#​30001).

Breaking Changes

• N/A

Deprecations

• N/A

Bugfixes

• Fixed a memory leak affecting some Go SDK since 2.46.0. (#​28142)

Security Fixes

• N/A

Known Issues

• N/A

List of Contributors

According to git shortlog, the following people contributed to the 2.54.0 release. Thank you to all contributors!

Ahmed Abualsaud

Alexey Romanenko

Anand Inguva

Andrew Crites

Arun Pandian

Bruno Volpato

caneff

Chamikara Jayalath

Changyu Li

Cheskel Twersky

Claire McGinty

clmccart

Damon

Danny McCormick

dependabot[bot]

Edward Cheng

Ferran Fernández Garrido

Hai Joey Tran

hugo-syn

Issac

Jack McCluskey

Jan Lukavský

JayajP

Jeffrey Kinard

Jerry Wang

Jing

Joey Tran

johnjcasey

Kenneth Knowles

Knut Olav Løite

liferoad

Marc

Mark Zitnik

martin trieu

Mattie Fu

Naireen Hussain

Neeraj Bansal

Niel Markwick

Oleh Borysevych

pablo rodriguez defino

Rebecca Szper

Ritesh Ghorse

Robert Bradshaw

Robert Burke

Sam Whittle

Shunping Huang

Svetak Sundhar

S. Veyrié

Talat UYARER

tvalentyn

Vlado Djerek

Yi Hu

Zechen Jian

v2.53.0: Beam 2.53.0 release

We are happy to present the new 2.53.0 release of Beam.
This release includes both improvements and new functionality.
See the download page for this release.

For more information on changes in 2.53.0, check out the detailed release notes.

Highlights

• Python streaming users that use 2.47.0 and newer versions of Beam should update to version 2.53.0, which fixes a known issue: (#​27330).

I/Os

• TextIO now supports skipping multiple header lines (Java) (#​17990).
• Python GCSIO is now implemented with GCP GCS Client instead of apitools (#​25676)
• Adding support for LowCardinality DataType in ClickHouse (Java) (#​29533).
• Added support for handling bad records to KafkaIO (Java) (#​29546)
• Add support for generating text embeddings in MLTransform for Vertex AI and Hugging Face Hub models.(#​29564)
• NATS IO connector added (Go) (#​29000).

New Features / Improvements

• The Python SDK now type checks collections.abc.Collections types properly. Some type hints that were erroneously allowed by the SDK may now fail. (#​29272)
• Running multi-language pipelines locally no longer requires Docker.
  Instead, the same (generally auto-started) subprocess used to perform the
  expansion can also be used as the cross-language worker.
• Framework for adding Error Handlers to composite transforms added in Java (#​29164).
• Python 3.11 images now include google-cloud-profiler (#​29561).

Breaking Changes

• Upgraded to go 1.21.5 to build, fixing CVE-2023-45285 and CVE-2023-39326

Deprecations

• Euphoria DSL is deprecated and will be removed in a future release (not before 2.56.0) (#​29451)

Bugfixes

• (Python) Fixed sporadic crashes in streaming pipelines that affected some users of 2.47.0 and newer SDKs (#​27330).
• (Python) Fixed a bug that caused MLTransform to drop identical elements in the output PCollection (#​29600).

List of Contributors

According to git shortlog, the following people contributed to the 2.53.0 release. Thank you to all contributors!

Ahmed Abualsaud

Ahmet Altay

Alexey Romanenko

Anand Inguva

Arun Pandian

Balázs Németh

Bruno Volpato

Byron Ellis

Calvin Swenson Jr

Chamikara Jayalath

Clay Johnson

Damon

Danny McCormick

Ferran Fernández Garrido

Georgii Zemlianyi

Israel Herraiz

Jack McCluskey

Jacob Tomlinson

Jan Lukavský

JayajP

Jeffrey Kinard

Johanna Öjeling

Julian Braha

Julien Tournay

Kenneth Knowles

Lawrence Qiu

Mark Zitnik

Mattie Fu

Michel Davit

Mike Williamson

Naireen

Naireen Hussain

Niel Markwick

Pablo Estrada

Radosław Stankiewicz

Rebecca Szper

Reuven Lax

Ritesh Ghorse

Robert Bradshaw

Robert Burke

Sam Rohde

Sam Whittle

Shunping Huang

Svetak Sundhar

Talat UYARER

Tom Stepp

Tony Tang

Vlado Djerek

Yi Hu

Zechen Jiang

clmccart

damccorm

darshan-sj

gabry.wu

johnjcasey

liferoad

lrakla

martin trieu

tvalentyn

v2.52.0: Beam 2.52.0 release

We are happy to present the new 2.52.0 release of Beam.
This release includes both improvements and new functionality.
See the download page for this release.

For more information on changes in 2.52.0, check out the detailed release notes.

Highlights

• Previously deprecated Avro-dependent code (Beam Release 2.46.0) has been finally removed from Java SDK "core" package.
  Please, use beam-sdks-java-extensions-avro instead. This will allow to easily update Avro version in user code without
  potential breaking changes in Beam "core" since the Beam Avro extension already supports the latest Avro versions and
  should handle this. (#​25252).
• Publishing Java 21 SDK container images now supported as part of Apache Beam release process. (#​28120)
  • Direct Runner and Dataflow Runner support running pipelines on Java21 (experimental until tests fully setup). For other runners (Flink, Spark, Samza, etc) support status depend on runner projects.

New Features / Improvements

• Add UseDataStreamForBatch pipeline option to the Flink runner. When it is set to true, Flink runner will run batch
  jobs using the DataStream API. By default the option is set to false, so the batch jobs are still executed
  using the DataSet API.
• upload_graph as one of the Experiments options for DataflowRunner is no longer required when the graph is larger than 10MB for Java SDK (PR#28621.
• state amd side input cache has been enabled to a default of 100 MB. Use --max_cache_memory_usage_mb=X to provide cache size for the user state API and side inputs. (Python) (#​28770).
• Beam YAML stable release. Beam pipelines can now be written using YAML and leverage the Beam YAML framework which includes a preliminary set of IO's and turnkey transforms. More information can be found in the YAML root folder and in the README.

Breaking Changes

• org.apache.beam.sdk.io.CountingSource.CounterMark uses custom CounterMarkCoder as a default coder since all Avro-dependent
  classes finally moved to extensions/avro. In case if it's still required to use AvroCoder for CounterMark, then,
  as a workaround, a copy of "old" CountingSource class should be placed into a project code and used directly
  (#​25252).
• Renamed host to firestoreHost in FirestoreOptions to avoid potential conflict of command line arguments (Java) (#​29201).

Bugfixes

• Fixed "Desired bundle size 0 bytes must be greater than 0" in Java SDK's BigtableIO.BigtableSource when you have more cores than bytes to read (Java) #​28793.
• watch_file_pattern arg of the RunInference arg had no effect prior to 2.52.0. To use the behavior of arg watch_file_pattern prior to 2.52.0, follow the documentation at https://beam.apache.org/documentation/ml/side-input-updates/ and use WatchFilePattern PTransform as a SideInput. (#​28948)
• MLTransform doesn't output artifacts such as min, max and quantiles. Instead, MLTransform will add a feature to output these artifacts as human readable format - #​29017. For now, to use the artifacts such as min and max that were produced by the eariler MLTransform, use read_artifact_location of MLTransform, which reads artifacts that were produced earlier in a different MLTransform (#​29016)
• Fixed a memory leak, which affected some long-running Python pipelines: #​28246.

Security Fixes

• Fixed CVE-2023-39325 (Java/Python/Go) (#​29118).
• Mitigated CVE-2023-47248 (Python) #​29392.

List of Contributors

According to git shortlog, the following people contributed to the 2.52.0 release. Thank you to all contributors!

Ahmed Abualsaud
Ahmet Altay
Aleksandr Dudko
Alexey Romanenko
Anand Inguva
Andrei Gurau
Andrey Devyatkin
BjornPrime
Bruno Volpato
Bulat
Chamikara Jayalath
Damon
Danny McCormick
Devansh Modi
Dominik Dębowczyk
Ferran Fernández Garrido
Hai Joey Tran
Israel Herraiz
Jack McCluskey
Jan Lukavský
JayajP
Jeff Kinard
Jeffrey Kinard
Jiangjie Qin
Jing
Joar Wandborg
Johanna Öjeling
Julien Tournay
Kanishk Karanawat
Kenneth Knowles
Kerry Donny-Clark
Luís Bianchin
Minbo Bae
Pranav Bhandari
Rebecca Szper
Reuven Lax
Ritesh Ghorse
Robert Bradshaw
Robert Burke
RyuSA
Shunping Huang
Steven van Rossum
Svetak Sundhar
Tony Tang
Vitaly Terentyev
Vivek Sumanth
Vlado Djerek
Yi Hu
aku019
brucearctor
caneff
damccorm
ddebowczyk92
dependabot[bot]
dpcollins-google
edman124
gabry.wu
illoise
johnjcasey
jonathan-lemos
kennknowles
liferoad
magicgoody
martin trieu
nancyxu123
pablo rodriguez defino
tvalentyn

---

Configuration

📅 Schedule: Branch creation - "before 8am on Monday" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

---

• If you want to rebase/retry this PR, check this box

---

This PR has been generated by Mend Renovate. View repository job log here.

[sonarcloud]

Quality Gate passed

Issues
0 New issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud