Huge `make` output on build failures cause PIE to hang when `symfony/console` renders

[derickr]

I am trying to install the MongoDB extension for PHP 8.5(dev), with the following command:

/tmp/php-build/workspace/install/8.4dev/bin/php /tmp/pie.phar --with-php-config=/tmp/php-build/workspace/install/master/bin/php-config install mongodb/mongodb-extension:2.1.1 -vvv

I am using PHP 8.4 to run PIE, as the version I had, didn't yet work with PHP 8.5 (not relevant to the bug). I have tested with PHP 8.5 running PIE, and I am getting the same issue anyway.

MongoDB version 2.1.1 does not compile yet with PHP 8.5. That is fine, but at the end of the run, I get first the error:

In file included from /home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/phongo_execute.c:23:
/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/contrib/php_array_api.h: In function ‘php_array_zval_to_string’:
/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/contrib/php_array_api.h:353:36: error: implicit declaration of function ‘IS_INTERNED’; did you mean ‘IS_STR_INTERNED’? [-Wimplicit-function-declaration]
  353 |                         *pfree = ! IS_INTERNED(Z_STR(c));
      |                                    ^~~~~~~~~~~
      |                                    IS_STR_INTERNED
make: *** [Makefile:1038: src/phongo_execute.lo] Error 1
 cc -I. -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1 -I/tmp/php-build/workspace/install/master/include/php -I/tmp/php-build/workspace/install/master/include/php/main -I/tmp/php-build/workspace/install/master/include/php/TSRM -I/tmp/php-build/workspace/install/master/include/php/Zend -I/tmp/php-build/workspace/install/master/include/php/ext -I/tmp/php-build/workspace/install/master/include/php/ext/date/lib -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/libmongoc/src/common/src/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/libmongoc/src/uthash/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/libmongoc/src/libbson/src/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/libmongoc/src/libbson/src/jsonsl/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/libmongoc/src/libmongoc/src/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/libmongoc/src/utf8proc-2.8.0/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/libmongocrypt/src/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/libmongocrypt/kms-message/src/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/libmongocrypt-compat/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/BSON/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/MongoDB/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/MongoDB/Exception/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/MongoDB/Monitoring/ -I/home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/contrib/ -DHAVE_CONFIG_H -g -O0 -D_GNU_SOURCE -DZEND_COMPILE_DL_EXT=1 -c /home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/vendor/mongodb/mongodb-extension/mongodb-2.1.1/src/BSON/MinKeyInterface.c -MMD -MF src/BSON/MinKeyInterface.dep -MT src/BSON/MinKeyInterface.lo  -fPIC -DPIC -o src/BSON/.libs/MinKeyInterface.o

And then:

    Install of mongodb/mongodb-extension failed
Downloading https://packagist.org/downloads/
[201] https://packagist.org/downloads/

{cursor position}

And then it just hangs.

The versions etc are:

> Using PHP 8.4.13-dev
> PHP is using the following php.ini file:
  /tmp/php-build/workspace/install/8.4dev/lib/php.ini

> Checking Box requirements:
  ✔ This application requires a PHP version matching "8.1.*||8.2.*||8.3.*||8.4.*||8.5.*".
  ✔ The package "symfony/polyfill-mbstring" requires the extension "iconv".
  ✔ The package "justinrainbow/json-schema" requires the extension "json".
  ✔ The package "composer/ca-bundle" requires the extension "openssl".
  ✔ The package "composer/ca-bundle" requires the extension "pcre".
  ✔ The package "marc-mabe/php-enum" requires the extension "reflection".
  ✔ The package "symfony/service-contracts" conflicts with the extension "psr".
  
                                                                                                                                                                                                                                                                               
 [OK] Your system is ready to run the application.                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                               

🥧 PHP Installer for Extensions (PIE) 1.1.0@a4068ee, from The PHP Foundation
This command may need elevated privileges, and may prompt you for your password.
You are running PHP 8.4.13-dev
Target PHP installation: 8.5.0-dev nts, on Linux/OSX/etc x86_64 (from /tmp/php-build/workspace/install/master/bin/php)
Using pie.json: /home/xdebug-ci/.config/pie/php8.5_c09d9be149d9095f48146608e828f27f/pie.json

[asgrim]

Thanks @derickr definitely something up here.

Findings so far; there is some kind of slowness or potentially endless loop in \Symfony\Component\Console\Application::doRenderThrowable, I think around here:

https://github.com/symfony/symfony/blob/dd399b476c42d8437191e162de955a35a408d99f/src/Symfony/Component/Console/Application.php#L883-L891

I suspect it's choking on the size of the exception message (which is the whole make output, essentially); the message is 1681277 bytes long (~1.7M).

[derickr]

I can see if I can get you a PHP back trace with Xdebug's out-of-band initialiser. It did sit for tens of minutes here once, so I bet it's some infinity loop, or recursive nested structure.

[asgrim]

Yeah, I'm certain it is something to do with the size of the exception message. The following script (outside of PIE) exhibits the same behaviour:

<?php

require_once 'vendor/autoload.php';

$output = new Symfony\Component\Console\Output\ConsoleOutput();
$app = new \Symfony\Component\Console\Application();

$output->writeln('starting...');

$e = new Exception(file_get_contents('exception.txt'));
$app->renderThrowable($e, $output);

$output->writeln('ending...');

(where exception.txt contains the ~1.7mb exception message)

edit: the issue is not exclusive to PHP 8.5, it is just reproducible in PIE because of the build failure of Mongo in 8.5 :) will update the issue title accordingly

[asgrim]

I think a pragmatic solution to this issue would be to catch the exception, write the output to a file somewhere, cut it down to say the last 5 lines of output or something (and tell the user that the full output is in the file) - sorta like how npm handles long output. Any other suggestions/counter-offers welcome, and I'll start looking into this.

[derickr]

I think that makes sense, but I would probably use quite a few more lines than 5 (maybe 20?). Obviously, for me that wouldn't have added anything, as with -vvv, it already showed me all the output.