Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Questions about libpython size, specifically on Linux x86_64 #275

Open
ceejatec opened this issue Jun 21, 2024 · 6 comments
Open

Questions about libpython size, specifically on Linux x86_64 #275

ceejatec opened this issue Jun 21, 2024 · 6 comments

Comments

@ceejatec
Copy link

Several possibly-related questions. Note that I'm not necessarily saying anything is "wrong" here, I'm just trying to understand the differences.

So I've noticed two things about libpython.3.xx.so.1.0, working on x86_64-unknown-linux-gnu:

  1. Python 3.12's libpython is 4x the size of Python 3.11: 212MB vs. 58MB. If you strip both of them, the size difference largely vanishes; Python 3.12 is 25.3MB, where Python 3.11 is 24.4MB.
  2. python-build-standalone's stripped libpython is still something like 4x larger than the libpython that comes with Fedora 40, which is 6.1MB for Python 3.12 and 4.9MB for Python 3.11.

This is particularly noticeable when developing standalone binaries with PyInstaller. I've hacked PyInstaller locally to work with python-build-standalone, but the resulting binaries are nearly 3x the size compared to building with Fedora's python even if I build with the stripped libpython, which is kind of rough.

@jfcherng
Copy link

jfcherng commented Jun 28, 2024

  1. Python 3.12's libpython is 4x the size of Python 3.11: 212MB vs. 58MB. If you strip both of them, the size difference largely vanishes; Python 3.12 is 25.3MB, where Python 3.11 is 24.4MB.

Thanks for this information. I was wondering why my PyInstaller-packaged APP became much larger. 😄

@bulletmark
Copy link

See also related discussion here.

@jfcherng
Copy link

jfcherng commented Jun 28, 2024

See also related discussion here.

Thanks for this information. I do have a script for installation so I can add a strip step during installing. Currently I just use strip without any CLI flags. Do you have recommended flags?

@bulletmark
Copy link

My installer uses strip -p --strip-unneeded.

@indygreg
Copy link
Owner

I wonder if the debug symbols size has something to do with BOLT, which is 3.12+ only. Maybe BOLT is producing sub-optimal DWARF records, not preserving compression in symbols, or something along those lines.

@charliermarsh
Copy link
Collaborator

FWIW on macOS, I only see a ~12% reduction in libpython3.10.dylib size.

charliermarsh added a commit that referenced this issue Jul 21, 2024
## Summary

This PR adds an `install_only_stripped` variant, which is generated by taking the `install_only` variant and removing debug symbols.

Closes #277.
Closes #174.
Related to #275.

## Test Plan

On macOS:

- Downloaded [cpython-3.10.14+20240713-aarch64-apple-darwin-install_only.tar.gz](https://github.com/indygreg/python-build-standalone/releases/download/20240713/cpython-3.10.14+20240713-aarch64-apple-darwin-install_only.tar.gz) locally.
- Ran: `cargo run convert-install-only-stripped cpython-3.10.14+20240713-aarch64-apple-darwin-install_only.tar.gz`.
- Relocated `cpython-3.10.14+20240713-aarch64-apple-darwin-install_only_stripped.tar.gz` to another directory.
- Unzipped `cpython-3.10.14+20240713-aarch64-apple-darwin-install_only.tar.gz`.
- Ran `./python` in `python/python/bin`.

Performed the same procedure on Windows.
zanieb pushed a commit to zanieb/python-build-standalone that referenced this issue Sep 4, 2024
This PR adds an `install_only_stripped` variant, which is generated by taking the `install_only` variant and removing debug symbols.

Closes indygreg#277.
Closes indygreg#174.
Related to indygreg#275.

On macOS:

- Downloaded [cpython-3.10.14+20240713-aarch64-apple-darwin-install_only.tar.gz](https://github.com/indygreg/python-build-standalone/releases/download/20240713/cpython-3.10.14+20240713-aarch64-apple-darwin-install_only.tar.gz) locally.
- Ran: `cargo run convert-install-only-stripped cpython-3.10.14+20240713-aarch64-apple-darwin-install_only.tar.gz`.
- Relocated `cpython-3.10.14+20240713-aarch64-apple-darwin-install_only_stripped.tar.gz` to another directory.
- Unzipped `cpython-3.10.14+20240713-aarch64-apple-darwin-install_only.tar.gz`.
- Ran `./python` in `python/python/bin`.

Performed the same procedure on Windows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants