A stale function might get called when unloading shared libraries on FBSD

[dlangBugzillaToGithub]

Martin Nowak reported this on 2015-07-24T23:34:51Z

Transferred from https://issues.dlang.org/show_bug.cgi?id=14824

CC List

• ponce

Description

FreeBSD's runtime linker has a bug where it could resolve a PLT
function entry to a weak definition in a dynamically loaded shared
library without pinning that library. Once the lib get's unloaded
further calls to that function will crash.

This happens with the host test [¹] which loads plugin1.so and plugin2.so (both depending on libdruntime.so).
On the first call to dur!"usecs" the PLT entry is resolved to
plugin1.so b/c it preceeds libdruntime.so in the symbol search
order. When plugin1.so gets unloaded, the PLT of druntime would still point to the definition in plugin1, even though plugin2 might still call those functions in druntime.

GLIBC get's this right and pins plugin1.so, see [²].

This was found on FBSD-8.4, but a look at the relevant source code [³] indicates, that it's still not fixed.

A workaround is to explicitly load the common dependent library (libdruntime.so here) before loading the plugins, that way the symbol search order will prefer druntime.

[¹]: https://github.com/D-Programming-Language/druntime/blob/645edac3483c8908de29c55c9312dc5dcf2f6bdd/test/shared/src/host.c
[²]: https://sourceware.org/git/?p=glibc.git;a=blob;f=elf/dl-lookup.c;hb=6b183b833968010a95ba41ed307dd8bea7b2e641#l917
[³]: https://github.com/freebsd/freebsd/blob/527ac1e9fb2b2f29df0bcfb2e91053cea93956bb/libexec/rtld-elf/rtld.c#L3846

[dlangBugzillaToGithub]

aliloko commented on 2015-09-10T14:13:05Z

I'm hitting a very similar bug on OS X 10.10.4, let me explain.



== Setup ==

Here the host program source.

What it does is:
 - for each dynlib in command line:
    - load dynlibs
    - call the VSTPluginMain function if it exist, 
    - unload it

--------------------- ldvst.cpp --------------------

#include <dlfcn.h>
#include <cstdio>
#include <cstring>
#include <vector>

typedef __cdecl void* (*VSTPluginMain_t)(void*);

int main(int argc, char**argv)
{
    std::vector<char*> dllPaths;
    if (argc < 2)
    {
        printf("usage: ldvst [-lazy] <thing.vst>\n");
        return 1;
    }

    bool lazy = false;

    for (int i = 1; i < argc; ++i)
    {
        char* arg = argv[i];
        if (strcmp(arg, "-lazy") == 0)
            lazy = true;
        else if (strcmp(arg, "-now") == 0)
            lazy = false;
        else
            dllPaths.push_back(arg);
    }

    for (int i = 0; i < dllPaths.size(); ++i)
    {
        char* dllPath = dllPaths[i];

        printf("dlopen(%s)\n", dllPath);
        void* handle = dlopen(dllPath, lazy ? RTLD_LAZY : RTLD_NOW);
        if (handle == NULL)
        {
            printf("error: dlopen of %s failed\n", dllPath);
            return 2;
        }

        VSTPluginMain_t VSTPluginMain = (VSTPluginMain_t) dlsym(handle, "VSTPluginMain");
        printf("dlsym returned %p\n", (void*)VSTPluginMain);

        if (VSTPluginMain != NULL)
        {
            void* result = VSTPluginMain(NULL);
            printf("VSTPluginMain returned %p\n", result);
        }

        printf("dlclose(%s)\n\n", dllPath);
        dlclose(handle);
    }
    return 0;
}
-----------------------------------------------------

The host is compiled with:

$ clang++ ldvst.cpp -o ldvst-cpp


Here the whole dynlib source:

----------------- distort.d -------------------------

extern(C) void* VSTPluginMain(void* hostCallback)
{
    import core.runtime;
    Runtime.initialize();
    import std.stdio;
    import core.stdc.stdio;
    printf("Hello !\n");
    Runtime.terminate();
    return null;
}

------------------------------------------------------

This dynlib can be built with ldc:

$ ldc2 -shared -oflibdistort.so -g -w -I. distort.d -relocation-model=pic

or with dmd

$ dmd -c -ofdistort.o -debug -g -w -version=Have_distort -I. distort.d -fPIC
$ dmd -oflibdistort.so distort.o -shared -g


For the purpose of demonstration you need another C dynlib, for example 
/System/Library/Frameworks/Cocoa.framework/Cocoa on OS X.


Now the bug triggers when calling:


== How to reproduce ==

$ ldvst-cpp libdistort.so 
   => works

$ ldvst-cpp libdistort.so libdistort.so 
   => works

$ ldvst-cpp /System/Library/Frameworks/Cocoa.framework/Cocoa libdistort.so 
   => works

$ ldvst-cpp libdistort.so /System/Library/Frameworks/Cocoa.framework/Cocoa
   => FAIL, and that's precisely the case that happen in production :(

$ ldvst-cpp /System/Library/Frameworks/Cocoa.framework/Cocoa libdistort.so /System/Library/Frameworks/Cocoa.framework/Cocoa
   => works

In other words, if the host program loads a D dynlib first, then a C dynlib, then the second dlopen fail.

This has eluded me for a complete month.