Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Properly handle RPATH $ORIGIN #75

Merged
merged 8 commits into from
Mar 10, 2019
Merged

Properly handle RPATH $ORIGIN #75

merged 8 commits into from
Mar 10, 2019

Conversation

JonathonReinhart
Copy link
Owner

@JonathonReinhart JonathonReinhart commented Nov 19, 2018 •
edited
Loading

This makes a number of fixes which all revolve around using ldd to discover dependencies:

  • Run ldd on executables in their original locations, rather than after they're copied temp directory. This is important if they use RPATH which includes $ORIGIN
  • PyInstaller:
    • Extract all binaries before scanning for dependencies. This is important if there are dependencies present in the PyInstaller archive
    • Set LD_LIBRARY_PATH to the temporary directory where we extract files from the PyInstaller archive

Fixes #61

@JonathonReinhart
Copy link
Owner Author

JonathonReinhart commented Nov 19, 2018 •
edited
Loading

Note: I still need to test this on PyInstaller applications, like #61 referred to.

I'm pretty sure it won't work because pyinstaller.py calls get_shobj_deps(tmppath) where tmppath is not the original path. The same fix needs to be made there also.

@oz123
Copy link

oz123 commented Jan 9, 2019

@JonathonReinhart I just tested this branch with application built with pyinstaller. I can confirm that this fix won't work for pyinstaller bundles.

@JonathonReinhart JonathonReinhart mentioned this pull request Jan 11, 2019
Closed
@cakebake
Copy link

cakebake commented Feb 18, 2019 •
edited
Loading

I just tested this branch with application built with pyinstaller. I can confirm that this fix won't work for pyinstaller bundles.

I can confirm that too. What is the status here? Unfortunately I am not really in the topic in python.

btw: Thank you for this project ;)

Adding

sed -i 's/# Some shared objs might have no DT_NEEDED tags (see issue #67)/print("ldd, unexpected lin in ldd output: " + line)\n            continue/g' \
  /usr/local/lib/python3.7/dist-packages/staticx/elf.py

... results in

root@c41d9d88bc9c:/tmp# staticx /tmp/dist/weasyprint /workdir/dist/weasyprint
ldd, unexpected lin in ldd output: 	libjpeg-3fe7dfc0.so.9.3.0 => not found
ldd, unexpected lin in ldd output: 	libopenjp2-e366d6b0.so.2.1.0 => not found
ldd, unexpected lin in ldd output: 	libz-a147dcb0.so.1.2.3 => not found
ldd, unexpected lin in ldd output: 	libtiff-8267adfe.so.5.4.0 => not found
ldd, unexpected lin in ldd output: 	libwebp-db943ca4.so.7.0.3 => not found
ldd, unexpected lin in ldd output: 	libwebpmux-062931a1.so.3.0.3 => not found
ldd, unexpected lin in ldd output: 	libwebpdemux-cc0d17dd.so.2.0.5 => not found
ldd, unexpected lin in ldd output: 	statically linked
ldd, unexpected lin in ldd output: 	statically linked
ldd, unexpected lin in ldd output: 	statically linked
ldd, unexpected lin in ldd output: 	statically linked
ldd, unexpected lin in ldd output: 	libz-a147dcb0.so.1.2.3 => not found

Example

@JonathonReinhart
Add pyinstall-cffi test
afc02e8 
This should hopefully shake out bugs related to Pyinstaller + RPATH
@JonathonReinhart
Build rpath origin test app/libs
abcd77e
@JonathonReinhart
Run rpath-origin test
92c19e9
@JonathonReinhart
staticx: Run ldd on original, unmoved user program
dfa201b 
Libraries and applications can use RPATH $ORIGIN to indicate that their
dependencies are in a path relative to them. If staticx moves/copies the
application, then ld.so won't be able to find the dependencies.

Fixes #61
@JonathonReinhart
pyinstaller: Extract all binaries before adding dependencies
e60c8c3 
Some of the dependencies may not have been extracted yet.
@JonathonReinhart
staticx: Add libpath parameter to get_shobj_deps()
9cabf00
@JonathonReinhart
pyinstaller: Set libpath when determining shobj deps
f825778 
This emulates the behavior of the PyInstaller bootloader:

https://github.com/pyinstaller/pyinstaller/blob/v3.4/bootloader/src/pyi_utils.c#L808-L811

This is necessary because some shared objects may depend on other shared
objects in the PyInstaller archive. Without this change, Staticx would
still build a valid program (because there are no additional
dependencies in this case), but ldd would report a missing library.

Fixes additional cases reported in #61.
@JonathonReinhart
Add script to run all tests in test/ dir
a0b377b
@JonathonReinhart JonathonReinhart merged commit 4f5f7a2 into master Mar 10, 2019
@JonathonReinhart JonathonReinhart deleted the 61-fix-rpath-origin branch March 10, 2019 08:42
@JonathonReinhart JonathonReinhart mentioned this pull request Oct 3, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Unexpected failure when wrapping executable due to missing line in ldd output
3 participants
@JonathonReinhart @oz123 @cakebake