Skip to content

Commit

Permalink
Mention how to use sanitizer with libblastrampoline (#41522)
Browse files Browse the repository at this point in the history
* Mention LBT_USE_RTLD_DEEPBIND=0

* Add step-by-step build instruction

* Bump LBT version to v3.1.0

Co-authored-by: Elliot Saba <staticfloat@gmail.com>
  • Loading branch information
tkf and staticfloat authored Jul 9, 2021
1 parent 0faa881 commit f8d04b2
Show file tree
Hide file tree
Showing 3 changed files with 104 additions and 33 deletions.
64 changes: 32 additions & 32 deletions deps/checksums/blastrampoline
Original file line number Diff line number Diff line change
@@ -1,34 +1,34 @@
blastrampoline-23de7a09bf354fe6f655c457bab5bf47fdd2486d.tar.gz/md5/0c8016a6e30bc2237184b816b613d11c
blastrampoline-23de7a09bf354fe6f655c457bab5bf47fdd2486d.tar.gz/sha512/7b7dbb101cf05ac833a8e5b09f1eec8eb99f0caafbe80075751a5f7e0bfe03a3b19d11d3507dadd13f503cfa9fc1a6cd53f3461af7d5afb39ca385a0ee26120b
libblastrampoline.v3.0.4+0.aarch64-apple-darwin.tar.gz/md5/0ea485b558637933f039d8793b3aa6c6
libblastrampoline.v3.0.4+0.aarch64-apple-darwin.tar.gz/sha512/bc7d4edd90f0f01cbc23cdce38bf3fa550ce988790fc63d8c84951b884465cc4f2d591d91a119e5cf8ce6b3161da6e09dc3689d52b4e71175be0d21afba00319
libblastrampoline.v3.0.4+0.aarch64-linux-gnu.tar.gz/md5/a175724976338ad493b200194687f63a
libblastrampoline.v3.0.4+0.aarch64-linux-gnu.tar.gz/sha512/30164779760514f0395daad24e9c4c457b1428f9532a47c1d5cc64c5843f3c392505825c70fdd6f0953c4b064b1a9fdb0834d4901b88244f1b31b49cb9baacdf
libblastrampoline.v3.0.4+0.aarch64-linux-musl.tar.gz/md5/f73b62096c9b3a128749179be29e2503
libblastrampoline.v3.0.4+0.aarch64-linux-musl.tar.gz/sha512/7cb42503ad8d7103c1187e8ee163324cf28c54b38dde535f7ca5e1ef7c29be187a11d10cc451baebf74327d48010bb6b6638fcf5c31418f47f853bbea2d8b7f6
libblastrampoline.v3.0.4+0.armv6l-linux-gnueabihf.tar.gz/md5/436323431e54429f35128f6f404771bf
libblastrampoline.v3.0.4+0.armv6l-linux-gnueabihf.tar.gz/sha512/760a5c35c17c70f5ae0d8cb543a982435f999662382d78070cc2b16f11000ed03e5de60944da67f959ef978f35bb033981a0a65460dddfd0f9bc146ff34e4e87
libblastrampoline.v3.0.4+0.armv6l-linux-musleabihf.tar.gz/md5/370ba82c807bb041d7f015b4f5ee8c48
libblastrampoline.v3.0.4+0.armv6l-linux-musleabihf.tar.gz/sha512/79be4403f2c3afd58103a6bb14bf42087f3c9b6737ece01d6abc339844d6b4b5f3e42ab6dd9d418e7f949cd2c1bbd8b934193763c1eb261298d9a0ce04c82019
libblastrampoline.v3.0.4+0.armv7l-linux-gnueabihf.tar.gz/md5/3ea77545ee850be82d497753360671b7
libblastrampoline.v3.0.4+0.armv7l-linux-gnueabihf.tar.gz/sha512/4dd1562f003c846fb84454ad2f6a0d6ae02d9662974d360d2d5ce6422a2c22d1f80e84ca500571cf391689d614c352288e1e4ef51392eb8438990a0ed96e66ae
libblastrampoline.v3.0.4+0.armv7l-linux-musleabihf.tar.gz/md5/b12cdf35ebe3eaa4c71db76be589d860
libblastrampoline.v3.0.4+0.armv7l-linux-musleabihf.tar.gz/sha512/b389958b71c0e31d60c69492055c2fe3eb6cb1c2ae1e99c55fee7768dd1c40c07b8d25c6525fddb60806c4d98731a20f851c790ddb3c6030f16a82a9c89b3336
libblastrampoline.v3.0.4+0.i686-linux-gnu.tar.gz/md5/078924f3c57310602aca1ac0063236ac
libblastrampoline.v3.0.4+0.i686-linux-gnu.tar.gz/sha512/7c37fff276943d32a7dfa9beb819569000ffb067b53a3dffa1752a9f40b29a10f87193ea5fb49d8fa3ac17a812a9c897c3fe8f89294e9aed1bac6bb3cf87eaac
libblastrampoline.v3.0.4+0.i686-linux-musl.tar.gz/md5/eb7c4a8454e9e5bdf84d8ce36dc6c6ed
libblastrampoline.v3.0.4+0.i686-linux-musl.tar.gz/sha512/01580512c9d7827ac0218ca43a546e595da0c2588b0dd60a7538b7c3e91ffce9aba3572eb5b682caa6f159b3453954169fc3a17f440a5a3057ad31c8094105f3
libblastrampoline.v3.0.4+0.i686-w64-mingw32.tar.gz/md5/ce9c539cd80f4ed80a76fdff81a5d8ee
libblastrampoline.v3.0.4+0.i686-w64-mingw32.tar.gz/sha512/0fb5452b3ee1d9a78f74ad5a95bca1049b903f37eb6b64f47479592d87a24e2b12d1838a7f2c5b3cede7d5e4c5d76f599cf47cfc7559f7ef01ba34acbf60a832
libblastrampoline.v3.0.4+0.powerpc64le-linux-gnu.tar.gz/md5/10239f0598e486969d882001c458a148
libblastrampoline.v3.0.4+0.powerpc64le-linux-gnu.tar.gz/sha512/85bf61041c7c1e7efa6405c16892dac342a88f94a3c8a03696044ddf32a65e94217bfa09165cf532f2adbea3c8ee9ed3ecdf7401d5d3d4a886c29879fb6c4116
libblastrampoline.v3.0.4+0.x86_64-apple-darwin.tar.gz/md5/a6dec79d2442617c03a32b69a2a5b1bf
libblastrampoline.v3.0.4+0.x86_64-apple-darwin.tar.gz/sha512/493b522b86568b82e3011518bdfd234d3a1f04fecd687f2a44d3a6037ff00d45e122d700ad41b434c3aeab2f9668f04afcc999236958e137fe1ab8660187a719
libblastrampoline.v3.0.4+0.x86_64-linux-gnu.tar.gz/md5/69fa36c363904e71dc7bd5c08a63c0a8
libblastrampoline.v3.0.4+0.x86_64-linux-gnu.tar.gz/sha512/c5563205653fc2d8c3efbf1449d3777a202e902f611febf5d8973a8b6247e6229cb9e3c30d9dd84d0bcafaa0ea9b85f433d6b0806e1e755c8ec3ba51e2efa125
libblastrampoline.v3.0.4+0.x86_64-linux-musl.tar.gz/md5/32f12ef8e1ad8b8ed4338bb5666c10ae
libblastrampoline.v3.0.4+0.x86_64-linux-musl.tar.gz/sha512/dce53f14bbcab1fd4c1836308b13fc38de6043f62fe74fcada894a725efad520153375bebdea8a48b8ae9edfb813c2f03ec194361f4556e6ed51238fbe8919ae
libblastrampoline.v3.0.4+0.x86_64-unknown-freebsd.tar.gz/md5/75dfdd1264f58b8ef43021bdd0ff6656
libblastrampoline.v3.0.4+0.x86_64-unknown-freebsd.tar.gz/sha512/9a14a89c82a93e23dbbf226cfe58bca2847a3eb66a9e2f552dc1598d721b837ed8ce3f9bc617266591872169dcd3728ff7ab251882de4230d6ce00c6f791c821
libblastrampoline.v3.0.4+0.x86_64-w64-mingw32.tar.gz/md5/3bf231ab990147d315ed6c243e35135b
libblastrampoline.v3.0.4+0.x86_64-w64-mingw32.tar.gz/sha512/9ae053f60060dccf78c2d77716d0dc8d6720799e2d92410e0de1412c1c7ddaf8523405994a9f0949c795da2d68f85056e851e17f3fd0a2f2aa4dc0be32f9caf7
libblastrampoline.v3.1.0+0.aarch64-apple-darwin.tar.gz/md5/9a72574c810323ebe7b496266a5b3d90
libblastrampoline.v3.1.0+0.aarch64-apple-darwin.tar.gz/sha512/559c91374882a137ce8b9f008e9d815dfebc175f65ac07bf784d590e31a07f60302de3d498e5dcc3f409f59fc2c7fbd1fb32623d25ed65bcc31c35a74c11f260
libblastrampoline.v3.1.0+0.aarch64-linux-gnu.tar.gz/md5/95802755d3b6205c88163ba313662fac
libblastrampoline.v3.1.0+0.aarch64-linux-gnu.tar.gz/sha512/7e3198429602fdd1196a45007e08a9b4fef2d303f8c7d7ff423fd19dfa970519d7402e95ea9acb32e7cd236b2c2f162e864a08f6062a8990cf9cbf75271a4074
libblastrampoline.v3.1.0+0.aarch64-linux-musl.tar.gz/md5/f1251167736acfcbe7a3fe0398796702
libblastrampoline.v3.1.0+0.aarch64-linux-musl.tar.gz/sha512/0564e79d44f2a53604a3941b9b638865e8f3f5b2945e43c08643a6fd13e3162c8c6ed156bc7f9f6fc69d0b6b23a8abd493ce4dc847b78f4cfe4a0bf5583fe4f0
libblastrampoline.v3.1.0+0.armv6l-linux-gnueabihf.tar.gz/md5/fe48bd1c1c0db93db72ded0df968ff40
libblastrampoline.v3.1.0+0.armv6l-linux-gnueabihf.tar.gz/sha512/6958a727a25668ecdbc9b38e1562239df64900ac426b13b1949a68cfb6c481b7431a70764343a9c3e65c683b5083d15881e7c6d2adc9bc3450d0816f54fcf224
libblastrampoline.v3.1.0+0.armv6l-linux-musleabihf.tar.gz/md5/28be9a338c9ea2f5c1b82a6230ff8324
libblastrampoline.v3.1.0+0.armv6l-linux-musleabihf.tar.gz/sha512/53d4e5ba559872c9aac60cb995df6d9711617f728a22b1b8f8dfb3f7286c8bd62f24dcedf428b457e1062799e31b9a68a17b653daae2f01131aa86d0801bbb32
libblastrampoline.v3.1.0+0.armv7l-linux-gnueabihf.tar.gz/md5/c1f1dcd14e110b1723557b9c0932637d
libblastrampoline.v3.1.0+0.armv7l-linux-gnueabihf.tar.gz/sha512/b573de04784dfd7803616a968675ce314af2ca4429029d57186619bf1b10d83d32801bcb74dd82430cb33d53d97117b9dd6d5cd64bb5c3850148a2414a0499f3
libblastrampoline.v3.1.0+0.armv7l-linux-musleabihf.tar.gz/md5/f00cc93db3543dce1492314fd8670c20
libblastrampoline.v3.1.0+0.armv7l-linux-musleabihf.tar.gz/sha512/13569d8aed844c1fb48a4e125bc822588fb795707e247f14e1b25dcd4622204f5b7bb976e50cb0fe9d49f1a327a37c2da43ebf0d2b4aac4d5e911e7c49046847
libblastrampoline.v3.1.0+0.i686-linux-gnu.tar.gz/md5/867876ae938dff24a73f0f2f3ffc0544
libblastrampoline.v3.1.0+0.i686-linux-gnu.tar.gz/sha512/cd7426ccc7eb59c7367ec5f23813d2cc19aac62eb0c43a2f769270f16da8daaece7dbde78e1487dd9ab624eb6389683400ee5a0654e45d6ba4f13496b9330cb5
libblastrampoline.v3.1.0+0.i686-linux-musl.tar.gz/md5/57d1942e23432b6a99365170473be723
libblastrampoline.v3.1.0+0.i686-linux-musl.tar.gz/sha512/775eb115df4554a04cc13c407915287a3917307c97f5e04618315478a862cc2d75597ac391d05e7ccbeb901a9bebc89e1ebb9e608b4737229eef253204d6f058
libblastrampoline.v3.1.0+0.i686-w64-mingw32.tar.gz/md5/8548d8ffacfcf5708b8df80d3ec81ad9
libblastrampoline.v3.1.0+0.i686-w64-mingw32.tar.gz/sha512/2397a03aae3794b8376606b88f33e3dd4ee4801a2f8e0cff5f28fd16a24024acb4babe6c5e041c1b04ac3dca025160d78c85b37a8ae8948d5969b2f618ea9053
libblastrampoline.v3.1.0+0.powerpc64le-linux-gnu.tar.gz/md5/a6dad5de86fab27cbe80eee54d5ce995
libblastrampoline.v3.1.0+0.powerpc64le-linux-gnu.tar.gz/sha512/5b9a0632c89c5b6e2660a88212293cbfe707cfc4b6d86abd1887068843c42da6513f65ca9601fa78baef508877c9e682cbbddc88a513692373c3c373c6b2d939
libblastrampoline.v3.1.0+0.x86_64-apple-darwin.tar.gz/md5/03e1a3ee1647d5056cd459c49ab46e73
libblastrampoline.v3.1.0+0.x86_64-apple-darwin.tar.gz/sha512/de5e772876b4ebcdd2981d382c613849c79206adfcc1eeac03e7b4d146fb33cbffe2b66791b9a02ad216d1959aaa56cbd36e9a8fc3f633216141280a27ff5b92
libblastrampoline.v3.1.0+0.x86_64-linux-gnu.tar.gz/md5/d731c29b90a26b70a2d65d15a213be33
libblastrampoline.v3.1.0+0.x86_64-linux-gnu.tar.gz/sha512/558aac3893e1d4e15dfd48b8be0567774560b6cd063d46ab935dd20f62a1abfb6e89d549cb010aa1b9c30fdc86bbea43bc16921819a7e5446cc6a9f3c6c293ad
libblastrampoline.v3.1.0+0.x86_64-linux-musl.tar.gz/md5/9023d6dc4822e523cbd6071ca0f39557
libblastrampoline.v3.1.0+0.x86_64-linux-musl.tar.gz/sha512/34d13ddaded44c2bf48f61489650c638d51bc52e4cfee3bfd89dbdcd3895d3e5a862c88c05fa76a1bf29e410125209fde717ffede9b4f4a39705d1866848df62
libblastrampoline.v3.1.0+0.x86_64-unknown-freebsd.tar.gz/md5/b68495b29ad8b31e4a4888cbbbe15ec1
libblastrampoline.v3.1.0+0.x86_64-unknown-freebsd.tar.gz/sha512/8783ba5a2b8f9391d0429045cf369889dd99b7253bf6a259bc613dedd038722a0f556c13a9c9c38da7ffd1050ee768d3d614944726629c7aa9b16f56e5b4a9e1
libblastrampoline.v3.1.0+0.x86_64-w64-mingw32.tar.gz/md5/0a0a31950022a56de76698bf5b14bb08
libblastrampoline.v3.1.0+0.x86_64-w64-mingw32.tar.gz/sha512/dfc1797cfe64b09381227508d16872421af99f416ffe5c3703f0fed4a88f8d52b2dd7f811388442264fe3060012b7df55f6f2a2b08887b0a36307804a978a6da
71 changes: 71 additions & 0 deletions doc/src/devdocs/sanitizers.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ An easy solution is to have an dedicated build folder for providing a matching t
with `BUILD_LLVM_CLANG=1`. You can then refer to this toolchain from another build
folder by specifying `USECLANG=1` while overriding the `CC` and `CXX` variables.

The sanitizers error out when they detect a shared library being opened using `RTLD_DEEPBIND`
(ref: [google/sanitizers#611](https://github.com/google/sanitizers/issues/611)).
Since [libblastrampoline](https://github.com/staticfloat/libblastrampoline) by default
uses `RTLD_DEEPBIND`, we need to set the environment variable `LBT_USE_RTLD_DEEPBIND=0`
when using a sanitizer.

To use one of of the sanitizers set `SANITIZE=1` and then the appropriate flag for the sanitizer you
want to use.

Expand Down Expand Up @@ -43,6 +49,71 @@ can be reduced by specifying `fast_unwind_on_malloc=0` and `malloc_context_size=
of backtrace accuracy. For now, Julia also sets `detect_leaks=0`, but this should be removed in
the future.

### Example setup

#### Step 1: Install toolchain

Checkout a Git worktree (or create out-of-tree build directory) at
`$TOOLCHAIN_WORKTREE` and create a config file `$TOOLCHAIN_WORKTREE/Make.user`
with

```
USE_BINARYBUILDER_LLVM=1
BUILD_LLVM_CLANG=1
```

Run:

```sh
cd $TOOLCHAIN_WORKTREE
make -C deps install-llvm install-clang install-llvm-tools
```

to install toolchain binaries in `$TOOLCHAIN_WORKTREE/usr/tools`

#### Step 2: Build Julia with ASAN

Checkout a Git worktree (or create out-of-tree build directory) at
`$BUILD_WORKTREE` and create a config file `$BUILD_WORKTREE/Make.user` with

```
TOOLCHAIN=$(TOOLCHAIN_WORKTREE)/usr/tools
# use our new toolchain
USECLANG=1
override CC=$(TOOLCHAIN)/clang
override CXX=$(TOOLCHAIN)/clang++
export ASAN_SYMBOLIZER_PATH=$(TOOLCHAIN)/llvm-symbolizer
USE_BINARYBUILDER_LLVM=1
override SANITIZE=1
override SANITIZE_ADDRESS=1
# make the GC use regular malloc/frees, which are hooked by ASAN
override WITH_GC_DEBUG_ENV=1
# default to a debug build for better line number reporting
override JULIA_BUILD_MODE=debug
# make ASAN consume less memory
export ASAN_OPTIONS=detect_leaks=0:fast_unwind_on_malloc=0:allow_user_segv_handler=1:malloc_context_size=2
JULIA_PRECOMPILE=1
# tell libblastrampoline to not use RTLD_DEEPBIND
export LBT_USE_RTLD_DEEPBIND=0
```

Run:

```sh
cd $BUILD_WORKTREE
make debug
```

to build `julia-debug` with ASAN.

## Memory Sanitizer (MSAN)

For detecting use of uninitialized memory, you can use Clang's [memory sanitizer (MSAN)](http://clang.llvm.org/docs/MemorySanitizer.html)
Expand Down
2 changes: 1 addition & 1 deletion stdlib/libblastrampoline_jll/Project.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name = "libblastrampoline_jll"
uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"
version = "3.0.4+0"
version = "3.1.0+0"

[deps]
Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
Expand Down

2 comments on commit f8d04b2

@nanosoldier
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Executing the daily package evaluation, I will reply here when finished:

@nanosoldier runtests(ALL, isdaily = true)

@nanosoldier
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something went wrong when running your job:

ProcessExitedException(2)

cc @maleadt

Please sign in to comment.