-
Notifications
You must be signed in to change notification settings - Fork 10.5k
Remove dependency on libtensorflow_framework.so. #27029
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove dependency on libtensorflow_framework.so. #27029
Conversation
Remove libtensorflow_framework.so as a build product. This should reduce toolchain size.
@swift-ci Please test tensorflow |
@swift-ci Please clean test tensorflow |
Please hold off on merging until toolchain building is tested locally. |
Toolchain building succeeded on macOS. |
Bazel produces multiple TensorFlow library artifacts: ``` $ ls -alh tensorflow/bazel-bin/tensorflow 18B libtensorflow.so -> libtensorflow.so.1 23B libtensorflow.so.1 -> libtensorflow.so.1.14.0 277M libtensorflow.so.1.14.0 ``` Previously, TensorFlow libraries were copied using `cp -a` via a glob pattern. For some reason, symlinks were not preserved: ``` $ ls -alh <TOOLCHAIN_BEFORE>.xctoolchain/usr/lib/swift/macosx 18B libtensorflow.so -> libtensorflow.so.1 277M libtensorflow.so.1 277M libtensorflow.so.1.14.0 # duplicate library ``` Now, rather than copying all libraries via glob, `copy_file_preserving_symlinks` copies exactly the necessary source files to the destination while preserving symlinks: ``` $ ls -alh <TOOLCHAIN_AFTER>.xctoolchain/usr/lib/swift/macosx 18B libtensorflow.so -> libtensorflow.so.1 277M libtensorflow.so.1 ``` This combined with removing libtensorflow_framework.so dependency (swiftlang#27029) led to a macOS toolchain size reduction from 4.91 GB to 3.45 GB.
…27032) Bazel produces multiple TensorFlow library artifacts: ``` $ ls -alh tensorflow/bazel-bin/tensorflow 18B libtensorflow.so -> libtensorflow.so.1 23B libtensorflow.so.1 -> libtensorflow.so.1.14.0 277M libtensorflow.so.1.14.0 ``` Previously, TensorFlow libraries were copied using `cp -a` via a glob pattern. For some reason, symlinks were not preserved: ``` $ ls -alh <TOOLCHAIN_BEFORE>.xctoolchain/usr/lib/swift/macosx 18B libtensorflow.so -> libtensorflow.so.1 277M libtensorflow.so.1 277M libtensorflow.so.1.14.0 # duplicate library ``` Now, rather than copying all libraries via glob, `copy_file_preserving_symlinks` copies exactly the necessary source files to the destination while preserving symlinks: ``` $ ls -alh <TOOLCHAIN_AFTER>.xctoolchain/usr/lib/swift/macosx 18B libtensorflow.so -> libtensorflow.so.1 277M libtensorflow.so.1 ``` This combined with removing libtensorflow_framework.so dependency (#27029) led to a macOS toolchain size reduction from 4.91 GB to 3.45 GB.
Remove libtensorflow_framework.so as a build product. This reduces toolchain size. Follow-up attempt to #20898.
Bazel produces multiple TensorFlow library artifacts: ``` $ ls -alh tensorflow/bazel-bin/tensorflow 18B libtensorflow.so -> libtensorflow.so.1 23B libtensorflow.so.1 -> libtensorflow.so.1.14.0 277M libtensorflow.so.1.14.0 ``` Previously, TensorFlow libraries were copied via `cp -p`. This did not preserve symlinks, leading to library duplication: ``` $ ls -alh <TOOLCHAIN_BEFORE>.xctoolchain/usr/lib/swift/macosx 18B libtensorflow.so -> libtensorflow.so.1 277M libtensorflow.so.1 277M libtensorflow.so.1.14.0 # duplicate library ``` Now, `cp -a` is used, which preserves symlinks: ``` $ ls -alh <TOOLCHAIN_AFTER>.xctoolchain/usr/lib/swift/macosx 18B libtensorflow.so -> libtensorflow.so.1 277M libtensorflow.so.1 ``` This combined with removing libtensorflow_framework.so dependency (swiftlang#27029) led to a macOS toolchain size reduction from 3.99 GB to 3.62 GB on the tensorflow-0.5 branch.
) Bazel produces multiple TensorFlow library artifacts: ``` $ ls -alh tensorflow/bazel-bin/tensorflow 18B libtensorflow.so -> libtensorflow.so.1 23B libtensorflow.so.1 -> libtensorflow.so.1.14.0 277M libtensorflow.so.1.14.0 ``` Previously, TensorFlow libraries were copied via `cp -p`. This did not preserve symlinks, leading to library duplication: ``` $ ls -alh <TOOLCHAIN_BEFORE>.xctoolchain/usr/lib/swift/macosx 18B libtensorflow.so -> libtensorflow.so.1 277M libtensorflow.so.1 277M libtensorflow.so.1.14.0 # duplicate library ``` Now, `cp -a` is used, which preserves symlinks: ``` $ ls -alh <TOOLCHAIN_AFTER>.xctoolchain/usr/lib/swift/macosx 18B libtensorflow.so -> libtensorflow.so.1 277M libtensorflow.so.1 ``` This combined with removing libtensorflow_framework.so dependency (#27029) led to a macOS toolchain size reduction from 3.99 GB to 3.62 GB on the tensorflow-0.5 branch.
) Bazel produces multiple TensorFlow library artifacts: ``` $ ls -alh tensorflow/bazel-bin/tensorflow 18B libtensorflow.so -> libtensorflow.so.1 23B libtensorflow.so.1 -> libtensorflow.so.1.14.0 277M libtensorflow.so.1.14.0 ``` Previously, TensorFlow libraries were copied via `cp -p`. This did not preserve symlinks, leading to library duplication: ``` $ ls -alh <TOOLCHAIN_BEFORE>.xctoolchain/usr/lib/swift/macosx 18B libtensorflow.so -> libtensorflow.so.1 277M libtensorflow.so.1 277M libtensorflow.so.1.14.0 # duplicate library ``` Now, `cp -a` is used, which preserves symlinks: ``` $ ls -alh <TOOLCHAIN_AFTER>.xctoolchain/usr/lib/swift/macosx 18B libtensorflow.so -> libtensorflow.so.1 277M libtensorflow.so.1 ``` This combined with removing libtensorflow_framework.so dependency (#27029) led to a macOS toolchain size reduction from 3.99 GB to 3.62 GB on the tensorflow-0.5 branch.
Remove
libtensorflow_framework.so
as a build product.This should reduce toolchain size.
Follow-up attempt to #20898.