Synk reporting security vulnerability in latest version of Apache-beam 2.52.0 Python SDK

[ff-sdesai]

What happened?

I am trying to add the dependency of apache-beam 2.52.0 (latest version) to pyproject.toml file. However, Synk is reporting a vulnerability during the build process in pyarrow 11.0.0 which Apache beam uses internally.

Pin pyarrow@11.0.0 to pyarrow@14.0.1 to fix
  ✗ Deserialization of Untrusted Data (new) [Critical Severity][https://security.snyk.io/vuln/SNYK-PYTHON-PYARROW-6052811] in pyarrow@11.0.0
    introduced by apache-beam@2.52.0 > pyarrow@11.0.0

I tried going to back to Apache beam 2.44.0 which uses pyarrow 9 internally but same vulnerability is being reported with all the versions. Is there any workaround for this?

Issue Priority

Priority: 0 (outage / urgent vulnerability)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[damccorm]

The vulnerability is patched via the pyarrow-hotfix package in #29396 but it is unsurprising that it still shows up on scanners since we do still have the old version of pyarrow present. #28410 tracks the work to fully upgrade pyarrow, but it is a non-trivial upgrade, so it may take a little longer. cc/ @tvalentyn @AnandInguva

I'm going to close this issue as a duplicate of #28410 - @ff-sdesai lets move any further conversation into that issue, if this is causing issues for you (even with the hotfix) that would be good to know/understand

[AnandInguva]

Note: Recently bumped the upper bound of pyarrow to include 14.0 but lower bound remains same due to compatibility with other package(such as TFT).

#29536

[damccorm]

Ah cool, so the full fix should be available with the next release (2.53.0). Thanks @AnandInguva

[ff-sdesai]

When is Apache beam 2.53.0 going to get released? Do we have a fixed date?

[damccorm]

The release follows the calendar here - https://calendar.google.com/calendar/u/0/embed?src=0p73sl034k80oob7seouanigd0@group.calendar.google.com&ctz=America/Los_Angeles - so the 2.53.0 release branch will get cut next Wednesday (December 13). After the branch is cut, it usually takes a few weeks to stabilize the release branch, validate the release, and successfully vote to promote it. So I would expect it to be released near the end of December or start of January.

You can track the release progress by subscribing to the beam dev list and following along with email updates, or checking in on the email list archive here - https://lists.apache.org/list.html?dev@beam.apache.org