Skip to content

Added position independent code flag for python compilation. #1258

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

pperanich
Copy link

In using the manylinux image to create wheels, the package I am compiling creates a shared library from the static python libraries in the image. Without the position independent code flag (-fPIC) I get the following error:

 /opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: buck-out/gen/third-party/python/python3.8_lib/libpython3.8.a(myreadline.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used    when making a shared object; recompile with -fPIC
 /opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: final link failed: nonrepresentable section on output
 collect2: error: ld returned 1 exit status

Adding the flag fixes this compilation issue. Since, as noted in #1250, the static libs remain in compressed form unless explicitly unpacked, this should have minimal affect on most users.

@auvipy auvipy requested a review from mayeut January 15, 2022 02:52
@pperanich
Copy link
Author

Let me know if I can provide any clarification on this PR. Thanks!

Copy link
Member

@mayeut mayeut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since, as noted in #1250, the static libs remain in compressed form unless explicitly unpacked, this should have minimal affect on most users.

This will impact all users at least indirectly. I expect python to be roughly 10% slower on i686 for example. This is clearly no showstopper. I'm analyzing other impacts (mostly size) and will give a feedback as soon as I have some figures.

The modification done here also has impact on other tools built (at least git, swig, maybe others).
The "-fPIC" added specifically in openssl build can be removed.

@mayeut
Copy link
Member

mayeut commented Jan 22, 2022

Let me know if I can provide any clarification on this PR.

A bit of background on the real use-case would be nice since it seems to differ from the one that lead to #1250

The PR does not change a thing regarding size on x86_64 so I think adding the flag is ok overall as -fPIC code can be linked in every case.

@pperanich, please address the comment about openssl build.

@rdb
Copy link
Contributor

rdb commented May 31, 2022

As the one who made the original PR to include the static libs, this PR seems like a very dangerous idea!

The only use case for including the static libs is to embed the interpreter into an executable. There are special linker mechanisms used when producing such an executable that make the symbols from libpython available to loaded extension modules.

Enabling -fPIC will allow libpython to be linked into a module library, which is exactly what people should not do. So it just opens up ways for people to shoot themselves into the foot.

Is there a legitimate use case for embedding the interpreter into a shared library? How could this even work, how can a library make its symbols globally available for subsequently dynamically loaded extension modules?

@rdb
Copy link
Contributor

rdb commented May 31, 2022

@pperanich Having had a cursory look at your use case, your project seems to be producing an extension module, in which case linking with libpython on Linux is not correct and would produce modules that are not compatible between different Python installations.

You should instead fix the build system not to link with libpython. I don't have the correct linker flags handy, but they should not be hard to find.

@mayeut
Copy link
Member

mayeut commented Nov 27, 2022

Per @rdb comment, the use-case involved seems to be an invalid one, so closing this PR.

@mayeut mayeut closed this Nov 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants