[Task]: Build RC and non-RC wheels in parallel.

[Abacn]

What happened?

code path: https://github.com/apache/beam/blob/master/.github/workflows/build_wheels.yml

There is exactly same steps "Build Wheel" and "Build RC wheels", both run the same command and produce same wheels with different naming:

8 wheels produced in 52 minutes:
  apache_beam-2.49.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl       13,728 kB
  apache_beam-2.49.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl   14,286 kB
  apache_beam-2.49.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl       14,585 kB
  apache_beam-2.49.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl   15,234 kB
  apache_beam-2.49.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl         14,369 kB
  apache_beam-2.49.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl     14,945 kB
  apache_beam-2.49.0-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl         13,999 kB
  apache_beam-2.49.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl     14,564 kB

8 wheels produced in 51 minutes:
  apache_beam-2.49.0rc1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl       13,728 kB
  apache_beam-2.49.0rc1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl   14,286 kB
  apache_beam-2.49.0rc1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl       14,585 kB
  apache_beam-2.49.0rc1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl   15,234 kB
  apache_beam-2.49.0rc1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl         14,369 kB
  apache_beam-2.49.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl     14,945 kB
  apache_beam-2.49.0rc1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl         13,999 kB
  apache_beam-2.49.0rc1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl     14,564 kB

Example run:

https://github.com/apache/beam/actions/runs/5467460207/jobs/9953842597

This becomes extremely time consuming as linux-aarch64 wheels costs 2.5 h to run, then the workflow costs >5 h to run.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[Abacn]

Looking into the workflow yml, apache-beam and apache-beam-rc artifacts are slightly different, on the version numbers, which means it has to be built twice:

beam/.github/workflows/build_wheels.yml

Line 154 in 79fcfc6

sed -i -e "s/${RELEASE_VERSION}/${RELEASE_VERSION}rc${RC_NUM}/g" apache_beam/version.py

Change this issue to FR