Description
I'm trying to track down a rare "zend_mm_heap corrupted" PHP-FPM error that occurs sometimes on our production servers, and I learned about AddressSanitizer yesterday. I've successfully managed to build Debian PHP packages with ASAN enabled by setting the environment variables documented on https://wiki.debian.org/LTS/Development/Asan before running debuild:
export DEB_CFLAGS_APPEND=-fsanitize=address
export DEB_CPPFLAGS_APPEND=-fsanitize=address
export DEB_CXXFLAGS_APPEND=-fsanitize=address
export DEB_LDFLAGS_APPEND='-fsanitize=address -static-libasan'
I know it works because simply running a php command will print a leak report:
martin.mein-iserv.de ~ # php -r 'echo "hello";'
hello
=================================================================
==1108212==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 4800 byte(s) in 1 object(s) allocated from:
#0 0x56416b5021d7 in __interceptor_calloc (/usr/bin/php8.2+0x3cf1d7)
#1 0x7f26d93f41f7 (<unknown module>)
Direct leak of 144 byte(s) in 6 object(s) allocated from:
#0 0x56416b50202f in __interceptor_malloc (/usr/bin/php8.2+0x3cf02f)
#1 0x7f26de638a4d (<unknown module>)
#2 0xc1e7e60b87e361ff (<unknown module>)
Direct leak of 64 byte(s) in 1 object(s) allocated from:
#0 0x56416b50202f in __interceptor_malloc (/usr/bin/php8.2+0x3cf02f)
#1 0x7f26e4951045 in gaih_inet ../sysdeps/posix/getaddrinfo.c:1058
#2 0x7f26e49527a4 in __GI_getaddrinfo ../sysdeps/posix/getaddrinfo.c:2256
#3 0x56416b4dd029 (/usr/bin/php8.2+0x3aa029)
#4 0x7f26dc641b74 (<unknown module>)
#5 0x7f26dc644025 (<unknown module>)
#6 0x7f26dc644162 (<unknown module>)
#7 0x7f26dc64419b (<unknown module>)
#8 0x7f26dc65af96 (<unknown module>)
#9 0x7f26dc75a6b7 (<unknown module>)
Direct leak of 48 byte(s) in 1 object(s) allocated from:
#0 0x56416b50202f in __interceptor_malloc (/usr/bin/php8.2+0x3cf02f)
#1 0x7f26dc63f54b (<unknown module>)
#2 0x7f26dc641c89 (<unknown module>)
#3 0x7f26dc644025 (<unknown module>)
#4 0x7f26dc644162 (<unknown module>)
#5 0x7f26dc64419b (<unknown module>)
#6 0x7f26dc65af96 (<unknown module>)
#7 0x7f26dc75a6b7 (<unknown module>)
Direct leak of 27 byte(s) in 1 object(s) allocated from:
#0 0x56416b50202f in __interceptor_malloc (/usr/bin/php8.2+0x3cf02f)
#1 0x7f26dc63f54b (<unknown module>)
#2 0x7f26dc65b4df (<unknown module>)
#3 0x7f26dc64416a (<unknown module>)
#4 0x7f26dc64419b (<unknown module>)
#5 0x7f26dc65af96 (<unknown module>)
#6 0x7f26dc75a6b7 (<unknown module>)
Direct leak of 8 byte(s) in 1 object(s) allocated from:
#0 0x56416b50202f in __interceptor_malloc (/usr/bin/php8.2+0x3cf02f)
#1 0x7f26dc8750f8 (<unknown module>)
Indirect leak of 48 byte(s) in 6 object(s) allocated from:
#0 0x56416b50202f in __interceptor_malloc (/usr/bin/php8.2+0x3cf02f)
#1 0x7f26de638a4d (<unknown module>)
#2 0xc1e7e60b87e361ff (<unknown module>)
Indirect leak of 27 byte(s) in 1 object(s) allocated from:
#0 0x56416b50202f in __interceptor_malloc (/usr/bin/php8.2+0x3cf02f)
#1 0x7f26e48f7bba in __GI___strdup string/strdup.c:42
SUMMARY: AddressSanitizer: 5166 byte(s) leaked in 18 allocation(s).
Unfortunately I have two problems which I am unsure how to solve:
-
For some reason, PHP seems to leak memory by default, which causes all PHP scripts now to print leak reports on STDERR. This breaks breaks a lot of our code that doesn't expect output from PHP scripts it calls. I'm generally not interested in the leak reports, but I haven't found a way to disable LSAN during compile time. I know I can disable it during runtime with the
ASAN_OPTIONSvariable, but that's hard to do system-wide, because many of our scripts will clear the environment for security reason before invoking other scripts. -
I have no way to reproduce the "zend_mm_heap corrupted" error in our PHP-FPM pools, but I know from our logs that it only occurs in two pools out of 23. Running the whole server with ASAN-enabled PHP-FPM works, but the slowdown is significant and I haven't yet dared to do that on a production server. Unfornuately, the way PHP(-FPM) works, I don't think that can easily run only those two pools with an ASAN-enabled build, while letting the rest run the normal non-ASAN-enabled build. I think that it might be useful to be able to compile PHP with ASAN disabled by default, and to only enable ASAN during runtime for those two pools. Is something like this technically possible?