Zstd builds fail with dfsan + hand written assembly

[terrelln]

We recently added some hand written assembly for a hot loop in zstd. After that our builds started failing with undefined symbol HUF_decompress4X2_usingDTable_internal_bmi2_asm_loop.dfsan errors. HUF_decompress4X2_usingDTable_internal_bmi2_asm_loop is our ASM function. This seems related to data flow sanitizer

We have the ability to disable the assembly implementation with a macro. E.g. by detecting __has_feature(dataflow_sanitizer) and disabling it. Is there a better way handle it?

[DavidKorczynski]

Am not sure, but is this an issue that happens in general or does the issue only happen in the OSS-Fuzz environment? If it's related to dfsan in general perhaps they will have better knowledge about it in the llvm repo: https://github.com/llvm/llvm-project/tree/main/compiler-rt/lib/dfsan

With that said, with limited knowledge of dfsan I could imagine it heavily relying on instrumenting LLVM IR and inline assembly will break that as dfsan can't instrument the assembly. Perhaps it's related to the ABI lists: https://clang.llvm.org/docs/DataFlowSanitizer.html#id5
Quote from the paragraph: "DataFlowSanitizer comes with a default ABI list which is intended to eventually cover the glibc library on Linux but it may become necessary for users to extend the ABI list in cases where a particular library or function cannot be instrumented (e.g. because it is implemented in assembly or another language which DataFlowSanitizer does not support)"

[terrelln]

Thanks for the advice! I've just disabled the ASM implementation when dfsan is enabled in facebook/zstd#2799.

[jonathanmetzman]

Thanks for the advice! I've just disabled the ASM implementation when dfsan is enabled in facebook/zstd#2799.

I would have just disabled dfsan. We aren't paying attention to it anymore, as it was Max Moroz who was doing that and he left Google.