msan false positive on FreeBSD in isprint() after setlocale()

[tavianator]

The following program:

$ cat foo.c
#include <ctype.h>
#include <limits.h>
#include <locale.h>
#include <stdio.h>

int main(void) {
        setlocale(LC_ALL, "C.UTF-8");
        return isprint(0x7F);
}

when compiled with clang16 -fsanitize=memory on FreeBSD:

$ clang16 --version
clang version 16.0.6
Target: x86_64-portbld-freebsd13.1
Thread model: posix
InstalledDir: /usr/local/llvm16/bin
$ clang16 -g -fsanitize=memory foo.c -o foo
$ elfctl -e +noaslr ./foo # Work around https://github.com/llvm/llvm-project/issues/53256

crashes with a use-of-uninitialized-value error that (I hope) is wrong:

$ ./foo
==21589==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x10bf805 in __sbmaskrune /usr/include/_ctype.h:109:2
    #1 0x10bf4c9 in __sbistype /usr/include/_ctype.h:122:12
    #2 0x10bf336 in main /usr/home/tavianator/code/bfs/foo.c:8:9

SUMMARY: MemorySanitizer: use-of-uninitialized-value /usr/include/_ctype.h:109:2 in __sbmaskrune
Exiting

[markjdb]

Did you recompile libc with MSan enabled? __sbmaskrune accesses an array, _CurrentRuneLocale->__runetype, that I would expect is initialized by libc.

[tavianator]

No, I assumed the interceptors would be enough. Typically msan need everything except libc to be instrumented.

[aokblast]

Unrelated to this issue, I read this patch and use the following command to build again in freebsd-src

make -DWITH_ASAN -DWITH_UBSAN KERNCONF=GENERIC-NODEBUG buildworld buildkernel -j4

However, it shows this error when building libc:

ld: error: cannot open /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/clang/16/lib/freebsd/libclang_rt.asan_static-x86_64.a: No such file or directory

I want to ask if I miss some compilation argument to build the world with sanitizer support?

[aokblast]

When I am building the world, I suffer from the following error:

ld: error: duplicate symbol: hexdump
>>> defined at sanitizer_common_interceptors.inc:10361 (/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:10361)
>>>            asan_interceptors.o:(__interceptor_hexdump) in archive /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/clang/16/lib/freebsd/libclang_rt.asan-x86_64.a
>>> defined at mpsutil.c:166 (/usr/src/usr.sbin/mpsutil/mpsutil.c:166)
>>>            mpsutil.o:(.text+0x1350)

I think hexdump is not the libc function thus it would't disable asan in the build-tool stage then finally cause this error.

I try to build mpsutil manually without asan, however, there are more libraries define their own hexdump and cause same error. Thus I think it is not a good workaround. Currently, I don't have any idea now, maybe we should erase hexdump interception in asan?

[aokblast]

I discover that hexdump is a library call on FreeBSD. So it will be hooked by msan. I don't know if there is any solution for this. Does @emaste have any idea if we want to fix this error?

[emaste]

I defer to @markjdb or @DimitryAndric

[markjdb]

I discover that hexdump is a library call on FreeBSD. So it will be hooked by msan. I don't know if there is any solution for this. Does @emaste have any idea if we want to fix this error?

This has been fixed in that LLVM no longer intercepts hexdump(). It didn't really make sense.

I've been doing a bit of work with MSan and it has a lot of problems with false positives. For instance, getc() isn't intercepted for some reason. Nor is cgetent(), used by ncurses.

If I don't instrument libc, then the interceptors cause problems. Consider fts_sort() in libc. It may reallocate an array (realloc is hooked by MSan), then populates the array, then sorts it with qsort (also hooked by MSan). MSan of course doesn't see that the array is initialized and raises a false positive.

I tried instrumenting libc with MSan, and that's a can of worms as well since it and libthr run a number of initializers before __msan_init() is called. Inline accesses to the shadow map triggers SIGSEGV and I don't see an LLVM option to disable that. I've tried selectively disabling instrumentation of certain libc and libthr files (e.g., we can't instrument getenv() or sysconf()), but it's fragile and tedious, and I haven't gotten it to work so far. Maybe MSan exposes a hook that would let libc map the shadow regions very early on, but I don't think so.