Add new scripts for building and cleaning wheels

release/BUILD

@@ -82,3 +82,26 @@ sh_binary(
         "//tensorflow_quantum/python/optimizers:spsa_minimizer",
     ],
 )
+
+exports_files([
+    "build_distribution.sh",
+    "clean_distribution.sh",
+])
+
+py_test(
+    name = "build_distribution_test",
+    srcs = ["build_distribution_test.py"],
+    data = [
+        "build_distribution.sh",
+    ],
+    python_version = "PY3",
+)
+
+py_test(
+    name = "clean_distribution_test",
+    srcs = ["clean_distribution_test.py"],
+    data = [
+        "clean_distribution.sh",
+    ],
+    python_version = "PY3",
+)

release/README.md

@@ -0,0 +1,112 @@
+# Tools for building releases of TensorFlow Quantum
+
+This directory contains configurations and scripts that the TensorFlow Quantum
+maintainers use to create Python packages for software releases. The process of
+making a TFQ release is complex, and has not been fully automated. The scripts
+in this directory help automate some steps and are a way of capturing the
+process more precisely, but there are still manual steps involved.
+
+## Background: how TensorFlow Quantum is linked with TensorFlow
+
+TFQ is implemented as a Python library that integrates static C++ objects. Those
+C++ objects are linked with TensorFlow static objects when both TFQ and
+TensorFlow are installed on your system. Unlike a pure Python library, the
+result is platform-dependent: the Python code itself remains portable, but the
+underlying C++ objects need to be compiled specifically for each target
+environment (operating system and CPU architecture).
+
+TensorFlow does not provide ABI stability guarantees between versions of
+TensorFlow. In order to avoid the need for users to compile the TFQ source code
+themselves when they want to install TFQ, each release of TFQ must be pinned to
+a specific version of TensorFlow. As a consequence, TFQ releases will not work
+with any other version of TensorFlow than the one they are pinned to.
+
+Python wheels for TFQ are produced by compiling them locally with a toolchain
+that matches that used by the version of TensorFlow being targeted by a given
+version of TFQ. A number of elements affect whether the whole process succeeds
+and the resulting wheel is portable to environments other than the specific
+computer TFQ is built on, including:
+
+*   The version of Python and the local Python environment
+*   The version of TensorFlow
+*   The TensorFlow build container used
+*   The Crosstool configuration used
+*   Whether CUDA is being used, and its version
+*   The dependency requirements implied by Cirq, NumPy, Protobuf, and
+    other Python packages
+
+## Procedure
+
+Building a TensorFlow Quantum release for Linux involves some additional steps
+beyond just building TFQ and producing an initial Python wheel. The procedure
+uses `auditwheel` to "repair" the resulting wheel; this improves the
+compatibility of the wheel so that it can run on a wider range of Linux
+distributions, even if those distributions have different versions of system
+libraries.
+
+The steps are:
+
+1.  Git clone the TensorFlow Quantum repo to a directory on your computer.
+
+1.  `cd` into the local clone directory.
+
+1.  Create a Python virtual environment.
+
+1.  Run `pip install -r requirements.txt`
+
+1.  Verify the major.minor version of Python you are using. The rest of these
+    instructions use 3.11 as an example.
+
+1.  Run `./release/build_distribution.sh -p 3.11`
+
+1.  If the above succeeds, it will leave the wheel in `/tmp/tensorflow_quantum/`
+    on your system. Take note of the name of the wheel file that
+    `build_distribution.sh` prints when it finishes.
+
+1.  Run `./release/clean_distribution.sh /tmp/tensorflow_quantum/WHEEL_FILE`,
+    where `WHEEL_FILE` is the file noted in the previous step. If this works, it
+    will create a new wheel file in `../wheelhouse`. If an error occurs, it will
+    hopefully report the problem. If the error is a platform tag mismatch, run
+    `./release/clean_distribution.sh -s /tmp/tensorflow_quantum/WHEEL_FILE`;
+    this will run auditwheel's `show` command on the wheel file to indicate what
+    version of `manylinux` this wheel can be made to run on if you use
+    `auditwheel` to repair it. With that information, you may be able to edit
+    the `build_distribution.sh` script to experiment with different values for
+    the Crosstool and/or the Docker images used.
+
+1.  Test the wheel. Go to a remotely hosted Colab (or any other Linux platform
+    that is distinctly difference from yours) upload this new generated wheel
+    file to local storage in the Colab, and test if it works. (E.g., try to run
+    `https://www.tensorflow.org/quantum/tutorials/hello_many_worlds` with the
+    new TensorFlow and TFQ wheels to verify things are working smoothly).
+
+1.  Repeat the `build_distribution.sh` and `clean_distribution.sh` steps for
+    different versions of Python.
+
+To make TFQ wheels that will work on other systems, it's essential that the
+platform tags of the TFQ wheel must match the tags of the current TensorFlow
+version you are targeting. (Visit `https://pypi.org/project/tensorflow/<target
+TF version>/#files` to determine what the tags are).
+
+## Testing the scripts
+
+There are some basic unit tests in the fils `build_distribution_test.py` and
+`clean_distribution_test.py`. They can be run using as follows:
+
+```shell
+# Move to the top-level directory of the repo before running the tests.
+cd ..
+bazel test //release:build_distribution_test //release:clean_distribution_test
+```
+
+## More information
+
+"TensorFlow SIG Build" is a community group dedicated to the TensorFlow build
+process. This repository is a showcase of resources, guides, tools, and builds
+contributed by the community, for the community. The following resources may be
+useful when trying to figure out how to make this all work.
+
+*   The "TF SIG Build Dockerfiles" document:
+    https://github.com/tensorflow/build/tree/ff4320fee2cf48568ebd2f476d7714438bfa0bee/tf_sig_build_dockerfiles#readme
+
+*   Other info in the SIG Build repo: https://github.com/tensorflow/build

release/build_distribution.sh

@@ -0,0 +1,159 @@
+#!/bin/bash
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# =============================================================================
+
+# Summary: build a wheel for TFQ using a TensorFlow SIG Build container.
+# Run this script with the option "-h" to get a usage summary.
+#
+# To ensure binary compatibility with TensorFlow, TFQ distributions are built
+# using TensorFlow's SIG Build containers and Crosstool C++ toolchain. This
+# script encapsulates the process. The basic steps this script performs are:
+#
+# 1. Write to a file a small shell script that does the following:
+#
+#    a) pip install TFQ's requirements.txt file
+#    b) run TFQ's configure.sh script
+#    c) run Bazel to build build_pip_package
+#    d) run the resulting build_pip_package
+#    e) copy the wheel created by build_pip_package to ./wheels
+#
+# 2. Start Docker with image tensorflow/build:${tf_version}-python${py_version}
+#    and run the script written in step 1.
+#
+# 3. Do some basic tests on the wheel using standard Python utilities.
+#
+# 4. Exit.
+
+set -eu
+
+# Find the top of the local TFQ git tree. Do it early in case this fails.
+thisdir=$(CDPATH="" cd -- "$(dirname -- "$0")" && pwd -P)
+repo_dir=$(git -C "${thisdir}" rev-parse --show-toplevel 2>/dev/null || \
+  echo "${thisdir}/..")
+
+# Default values for variables that can be changed via command line flags.
+tf_version="2.16"
+py_version=$(python3 --version | cut -d' ' -f2 | cut -d. -f1,2)
+cuda_version="12"
+cleanup="true"
+
+usage="Usage: ${0} [OPTIONS]
+Build a distribution wheel for TensorFlow Quantum.
+
+Configuration options:
+  -c X.Y  Use CUDA version X.Y (default: ${cuda_version})
+  -p X.Y  Use Python version X.Y (default: ${py_version})
+  -t X.Y  Use TensorFlow version X.Y (default: ${tf_version})
+
+General options:
+  -e     Don't run bazel clean at the end (default: do)
+  -n     Dry run: print commands but don't execute them
+  -h     Show this help message and exit"
+
+dry_run="false"
+while getopts "c:ehnp:t:" opt; do
+  case "${opt}" in
+    c) cuda_version="${OPTARG}" ;;
+    e) cleanup="false" ;;
+    h) echo "${usage}"; exit 0 ;;
+    n) dry_run="true" ;;
+    p) py_version="${OPTARG}" ;;
+    t) tf_version="${OPTARG}" ;;
+    *) echo "${usage}" >&2; exit 1 ;;
+  esac
+done
+shift $((OPTIND -1))
+
+# See https://hub.docker.com/r/tensorflow/build/tags for available containers.
+docker_image="tensorflow/build:${tf_version}-python${py_version}"
+
+# This should match what TensorFlow's .bazelrc file uses.
+crosstool="@sigbuild-r${tf_version}-clang_config_cuda//crosstool:toolchain"
+
+# Note: configure.sh is run inside the container, and it creates a .bazelrc
+# file that adds other cxxopt flags. They don't need to be repeated here.
+BUILD_OPTIONS="--cxxopt=-O3 --cxxopt=-msse2 --cxxopt=-msse3 --cxxopt=-msse4"
+
+# Create a script to be run by the shell inside the Docker container.
+build_script=$(mktemp /tmp/tfq_build.XXXXXX)
+trap 'rm -f "${build_script}" || true' EXIT
+
+# The printf'ed section dividers are to make it easier to search the output.
+cat <<'EOF' > "${build_script}"
+#!/bin/bash
+set -o errexit
+cd /tfq
+PREFIX='[DOCKER] '
+exec > >(sed "s/^/${PREFIX} /")
+exec 2> >(sed "s/^/${PREFIX} /" >&2)
+printf "Build configuration inside Docker container:\n"
+printf "  Docker image:     ${docker_image}\n"
+printf "  Crosstool:        ${crosstool}\n"
+printf "  TF version:       ${tf_version}\n"
+printf "  Python version:   ${py_version}\n"
+printf "  CUDA version:     ${cuda_version}\n"
+printf "  vCPUs available:  $(nproc)\n"
+printf "\n:::::::: Configuring Python environment ::::::::\n\n"
+python3 -m pip install --upgrade pip --root-user-action ignore
+pip install -r requirements.txt --root-user-action ignore
+printf "\n:::::::: Configuring TensorFlow Quantum build ::::::::\n\n"
+printf "Y\n" | ./configure.sh
+printf "\n:::::::: Starting Bazel build ::::::::\n\n"
+bazel build ${build_flags} release:build_pip_package
+printf "\n:::::::: Creating Python wheel ::::::::\n\n"
+bazel-bin/release/build_pip_package /build_output/
+if [ "${cleanup}" == "true" ]; then
+  printf "\n:::::::: Cleaning up ::::::::\n\n"
+  bazel clean --async
+fi
+EOF
+
+chmod +x "${build_script}"
+
+# Use 'set --' to build the command in the positional parameters ($1, $2, ...)
+set -- docker run -it --rm --network host \
+  -w /tfq \
+  -v "${repo_dir}":/tfq \
+  -v /tmp/tensorflow_quantum:/build_output \
+  -v "${build_script}:/tmp/build_script.sh" \
+  -e HOST_PERMS="$(id -u):$(id -g)" \
+  -e build_flags="--crosstool_top=${crosstool} ${BUILD_OPTIONS}" \
+  -e cuda_version="${cuda_version}" \
+  -e py_version="${py_version}" \
+  -e tf_version="${tf_version}" \
+  -e docker_image="${docker_image}" \
+  -e crosstool="${crosstool}" \
+  -e cleanup="${cleanup}" \
+  "${docker_image}" \
+  /tmp/build_script.sh
+
+if [ "${dry_run}" == "true" ]; then
+  # Loop through the positional parameters and simply print them.
+  printf "(Dry run) "
+  printf '%s ' "$@"
+  echo
+  echo '(Dry run) check-wheel-contents /tmp/tensorflow_quantum/*.whl'
+else
+  echo "Spinning up a Docker container with ${docker_image} …"
+  "$@"
+
+  # Do some basic checks on the wheel file.
+  echo "Doing basic sanity checks on the wheel …"
+  pip install -qq check-wheel-contents
+  check-wheel-contents /tmp/tensorflow_quantum/*.whl
+
+  echo "Done. Look for wheel in /tmp/tensorflow_quantum/."
+  ls -l /tmp/tensorflow_quantum/
+fi