Allow running position-independent executables (PIE)

[nyh]

On Linux we can compile a position-independent executable (PIE) - see -pie, -fpie compiler option. We should have no problem running such executables on OSv, because they are position-independent so can be relocated just like shared objects. But, unfortunately, Avi tried and noticed we're still missing a few features for it to work.

This issue is half-way to #190 (which is about running a normal Linux executable): It will be simpler to implement, and will allow us to run the same executable on both Linux and OSv, but we will not be able to take any random Linux executable and run it (it will have to be an executable compiled as PIE).

[nyh]

I just compiled memcached as "-pie" instead of "-shared", and the resulting exectuable seems to be working well in OSv (as well as in Linux, which is nice, because the ".so" can't run directly in Linux, but PIE does). Could it be that whatever problems we had with running PIEs were already solved?

One thing to note, however, is that even for PIE, what we do is to run their main() function. Technically, this is probably not the right thing to do - I am guessing we need to jump to some fixed starting point, not find a function called "main". But I think the current state, of calling main() should be good enough for 99% of the use cases.

So should we start using PIE instead of ".so", and after a bit more testing close this bug?

[nyh]

I just changed the memcached and rogue modules to compile with "-pie" instead of "-shared", and they seem to be working fine. So I'm closing this bug which claims PIE don't work, and we should instead open new bugs on specific things which don't work, if and when we find those.

[pdziepak]

TLS models may be a problem here. When an executable is linked local exec model is used. That will cause problems because there can be only one object using local exec model and that object is the kernel itself. There is -ftls-model flag but it looks like it only is a hint to specify stricter model than the linker normally would use, for example initial exec will be ok for shared objects in most cases (if OSv supported that). Unfortunately, when the specified model is more general than what would be normally used the flag is ignored.

[nyh]

On Fri, Jun 20, 2014 at 4:21 PM, Paweł Dziepak notifications@github.com
wrote:

TLS models may be a problem here. When an executable is linked local exec
model is used. That will cause problems because there can be only one
object using local exec model and that object is the kernel itself.

Are you saying that the "-pie" option changes the default TLS model? Do you
have a reference to that?
The gcc(1) manual page claims that

"The default without -fpic is "initial-exec"; with -fpic the default is
"global-dynamic".

And doesn't say anything about "-pie" changing this....
Note that objects belonging PIE executables are always compiled with -fpic,
exactly like objects belonging to a shared object, so how does the final
linking ruin the TLS handling which I thought was already produced during
the compilation stage?

I have to admit that I'm not (enough) of a dynamic linker expert to
understand the problem you're alluding to. Can you please try to create a
test case which demonstrates that a .so (linked with "-shared") runs
correctly, but a PIE (linked with "-pie") doesn't? Thanks.

There is -ftls-model flag but it looks like it only is a hint to specify
stricter model than the linker normally would use, for example initial exec
will be ok for shared objects in most cases (if OSv supported that).
Unfortunately, when the specified model is more general than what would be
normally used the flag is ignored.

—
Reply to this email directly or view it on GitHub
#213 (comment)
.

Nadav Har'El
nyh@cloudius-systems.com

[pdziepak]

2014-06-20 22:43 GMT+02:00 nyh notifications@github.com:

On Fri, Jun 20, 2014 at 4:21 PM, Paweł Dziepak notifications@github.com
wrote:

TLS models may be a problem here. When an executable is linked local
exec
model is used. That will cause problems because there can be only one
object using local exec model and that object is the kernel itself.

Are you saying that the "-pie" option changes the default TLS model? Do
you
have a reference to that?
The gcc(1) manual page claims that

"The default without -fpic is "initial-exec"; with -fpic the default is
"global-dynamic".

Apparently, the manual is wrong. If an executable is produced initial-exec
model is used, doesn't matter whether it is position independent or not.

And doesn't say anything about "-pie" changing this....
Note that objects belonging PIE executables are always compiled with
-fpic,
exactly like objects belonging to a shared object, so how does the final
linking ruin the TLS handling which I thought was already produced during
the compilation stage?

I have to admit that I'm not (enough) of a dynamic linker expert to
understand the problem you're alluding to. Can you please try to create a
test case which demonstrates that a .so (linked with "-shared") runs
correctly, but a PIE (linked with "-pie") doesn't? Thanks.

#include

thread_local int foo = 123;

int main()
{
std::cout << foo << "\n";
}

Works with -shared -fpic, doesn't work with -pie (with neither -fpic nor
-fpie) as local exec model is used.

There is -ftls-model flag but it looks like it only is a hint to specify
stricter model than the linker normally would use, for example initial
exec
will be ok for shared objects in most cases (if OSv supported that).
Unfortunately, when the specified model is more general than what would
be
normally used the flag is ignored.

[pdziepak]

Actually, GCC manual states that:

-pie
Produce a position independent executable on targets that support it. For predictable results, you must also specify the same set of options used for compilation (-fpie, -fPIE, or model suboptions) when you specify this linker option.

So, it looks like combination -pie -fpic isn't really correct, but that doesn't changes much. (Well, my earlier accusation of GCC manual being wrong is not really valid).

For -fpie we have:

-fpie
These options are similar to -fpic and -fPIC, but generated position independent code can be only linked into executables.

This allows GCC to use local exec model.

[nyh]

Hi Pawel, you're right about this TLS bug, I'll create a separate, specific
issue about this - no need to leave an issue which stated generally that
"PIE don't work" on OSv - they do, but have specific bugs that need to be
solved.

On Fri, Jun 20, 2014 at 5:19 PM, Paweł Dziepak notifications@github.com
wrote:

Actually, GCC manual states that:

-pie
Produce a position independent executable on targets that support it. For
predictable results, you must also specify the same set of options used for
compilation (-fpie, -fPIE, or model suboptions) when you specify this
linker option.

So, it looks like combination -pie -fpic isn't really correct, but that
doesn't changes much. (Well, my earlier accusation of GCC manual being
wrong is not really valid).

This is very interesting. I noticed that when you link with "-pie",
compiling without "-fpic" did not work, but I didn't think of compiling
with "-fpie" instead of "-fpic".

Do you have any idea how "-fpie" could possibly differ from "-fpic"?

For -fpie we have:

-fpie
These options are similar to -fpic and -fPIC, but generated position
independent code can be only linked into executables.

This allows GCC to use local exec model.

In practice, however, it doesn't seem to change anything... If I compile
with either -fpic or -fpie, the resulting test works correctly on Linux,
but doesn't work correctly on OSv.

[nyh]

On Sun, Jun 22, 2014 at 4:11 AM, Nadav Har'El nyh@cloudius-systems.com
wrote:

Do you have any idea how "-fpie" could possibly differ from "-fpic"?

I found a good (hopefully) explanation in
http://www.openbsd.org/papers/nycbsdcon08-pie/mgp00004.html

In practice, however, it doesn't seem to change anything... If I compile
with either -fpic or -fpie, the resulting test works correctly on Linux,
but doesn't work correctly on OSv.

Nadav Har'El
nyh@cloudius-systems.com