Segmentation fault with forks pragma

[mroemer]

When I use DBD::Oracle (v1.83) with forks (which we use to get consistent fork behavior with Apache/ModPerl) with Perl v5.34.0 on Ubuntu 22.04, we get a segmentation fault on any subsequent system() or exec() call but on when we use backticks.

use forks;
use DBI;

my $dbh = DBI->connect( 'dbi:Oracle://localhost', '', '',
    {
        ora_verbose => 6,
    });
$dbh->prepare('select 1 from dual');
$dbh->disconnect();
system("echo 2");

sleep 1;
print "Done!\n";

produces

$ perl -MDebug::DumpCore IPS/test_dbd_forks.pl 
2
ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] [si_int=0] [si_ptr=(nil)] [si_addr=0x594]
kpedbg_dmp_stack()+396<-kpeDbgCrash()+204<-kpeDbgSignalHandler()+113<-skgesig_sigactionHandler()+258<-__sighandler()<-Perl_csighandler3()+64<-__sighandler()<-0x00007FBFDC6F9117
Segmentation fault (core dumped)

With backticks, it works:

```perl
use forks;
use DBI;

my $dbh = DBI->connect( 'dbi:Oracle://localhost', '', '',
    {
        ora_verbose => 6,
    });
$dbh->prepare('select 1 from dual');
$dbh->disconnect();
print `echo 2`;

sleep 1;
print "Done!\n";

produces

$ perl -MDebug::DumpCore IPS/test_dbd_forks.pl 
2
Done!

I created a backtrace of the segmentation fault with gdb:

(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=11, threadid=139933470586368) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=11, threadid=139933470586368) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=139933470586368, signo=signo@entry=11) at ./nptl/pthread_kill.c:89
#3  0x00007f44d2042476 in __GI_raise (sig=11) at ../sysdeps/posix/raise.c:26
#4  0x00007f44cfcf396f in skgesigOSCrash () from /usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#5  0x00007f44d03f384d in kpeDbgSignalHandler () from /usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#6  0x00007f44cfcf3c82 in skgesig_sigactionHandler () from /usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#7  <signal handler called>
#8  0x000055f27d2fc820 in Perl_csighandler3 ()
#9  <signal handler called>
#10 __futex_abstimed_wait_common64 (private=1179670597, cancel=true, abstime=0x7f44ccce6d80, op=137, expected=0, futex_word=0x55f27fe7b4e8) at ./nptl/futex-internal.c:57
#11 __futex_abstimed_wait_common (cancel=true, private=1179670597, abstime=0x7f44ccce6d80, clockid=0, expected=0, futex_word=0x55f27fe7b4e8) at ./nptl/futex-internal.c:87
#12 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55f27fe7b4e8, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x7f44ccce6d80, 
    private=private@entry=0) at ./nptl/futex-internal.c:139
#13 0x00007f44d2093e9b in __pthread_cond_wait_common (abstime=0x7f44ccce6d80, clockid=1, mutex=0x55f280009f20, cond=0x55f27fe7b4c0) at ./nptl/pthread_cond_wait.c:503
#14 ___pthread_cond_timedwait64 (cond=0x55f27fe7b4c0, mutex=0x55f280009f20, abstime=0x7f44ccce6d80) at ./nptl/pthread_cond_wait.c:652
#15 0x00007f44cceabe53 in sltspctimewait () from /usr/lib/oracle/21/client64/lib/libclntshcore.so.21.1
#16 0x00007f44cd8bba09 in kpucpincrtime () from /usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#17 0x00007f44d2094ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#18 0x00007f44d2126850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

[mroemer]

The problem can be solved(?) by defining the environment variable THREADS_SIGCHLD_IGNORE.

$THREADS_SIGCHLD_IGNORE=1 perl -MDebug::DumpCore IPS/test_dbd_forks.pl
2
Done!

Not sure if this has any negative side effects.

From the forks documentation:

In order to be compatible with perl's core system() function on all platforms, extra care has gone into implementing a smarter $SIG{CHLD} in forks.pm. The only functional effect is that you will never need to (or be able to) reap threads (processes) if you define your own CHLD handler.

You may define the environment variable THREADS_SIGCHLD_IGNORE to force forks to use 'IGNORE' on systems where a custom CHLD signal handler has been automatically installed to support correct exit code of perl core system() function. Note that this should not be necessary unless you encounter specific issues with the forks.pm CHLD signal handler.

[ancientwizard]

When I use DBD::Oracle (v1.83) with forks (which we use to get consistent fork behavior with Apache/ModPerl) with Perl v5.34.0 on Ubuntu 22.04, we get a segmentation fault on any subsequent system() or exec() call but on when we use backticks.

use forks;
use DBI;

my $dbh = DBI->connect( 'dbi:Oracle://localhost', '', '',
{
ora_verbose => 6,
});
$dbh->prepare('select 1 from dual');
$dbh->disconnect();
system("echo 2");

sleep 1;
print "Done!\n";
produces

$ perl -MDebug::DumpCore IPS/test_dbd_forks.pl
2
ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] [si_int=0] [si_ptr=(nil)] [si_addr=0x594]
kpedbg_dmp_stack()+396<-kpeDbgCrash()+204<-kpeDbgSignalHandler()+113<-skgesig_sigactionHandler()+258<-__sighandler()<-Perl_csighandler3()+64<-__sighandler()<-0x00007FBFDC6F9117
Segmentation fault (core dumped)
With backticks, it works:

use forks;
use DBI;

my $dbh = DBI->connect( 'dbi:Oracle://localhost', '', '',
    {
        ora_verbose => 6,
    });
$dbh->prepare('select 1 from dual');
$dbh->disconnect();
print `echo 2`;

sleep 1;
print "Done!\n";
produces

$ perl -MDebug::DumpCore IPS/test_dbd_forks.pl 
2
Done!
I created a backtrace of the segmentation fault with `gdb`:

(gdb) bt
#0 __pthread_kill_implementation (no_tid=0, signo=11, threadid=139933470586368) at ./nptl/pthread_kill.c:44
#1 __pthread_kill_internal (signo=11, threadid=139933470586368) at ./nptl/pthread_kill.c:78
#2 __GI___pthread_kill (threadid=139933470586368, signo=signo@entry=11) at ./nptl/pthread_kill.c:89
#3 0x00007f44d2042476 in __GI_raise (sig=11) at ../sysdeps/posix/raise.c:26
#4 0x00007f44cfcf396f in skgesigOSCrash () from /usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#5 0x00007f44d03f384d in kpeDbgSignalHandler () from /usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#6 0x00007f44cfcf3c82 in skgesig_sigactionHandler () from /usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#7
#8 0x000055f27d2fc820 in Perl_csighandler3 ()
#9
#10 __futex_abstimed_wait_common64 (private=1179670597, cancel=true, abstime=0x7f44ccce6d80, op=137, expected=0, futex_word=0x55f27fe7b4e8) at ./nptl/futex-internal.c:57
#11 __futex_abstimed_wait_common (cancel=true, private=1179670597, abstime=0x7f44ccce6d80, clockid=0, expected=0, futex_word=0x55f27fe7b4e8) at ./nptl/futex-internal.c:87
#12 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55f27fe7b4e8, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x7f44ccce6d80,
private=private@entry=0) at ./nptl/futex-internal.c:139
#13 0x00007f44d2093e9b in __pthread_cond_wait_common (abstime=0x7f44ccce6d80, clockid=1, mutex=0x55f280009f20, cond=0x55f27fe7b4c0) at ./nptl/pthread_cond_wait.c:503
#14 ___pthread_cond_timedwait64 (cond=0x55f27fe7b4c0, mutex=0x55f280009f20, abstime=0x7f44ccce6d80) at ./nptl/pthread_cond_wait.c:652
#15 0x00007f44cceabe53 in sltspctimewait () from /usr/lib/oracle/21/client64/lib/libclntshcore.so.21.1
#16 0x00007f44cd8bba09 in kpucpincrtime () from /usr/lib/oracle/21/client64/lib/libclntsh.so.21.1
#17 0x00007f44d2094ac3 in start_thread (arg=) at ./nptl/pthread_create.c:442
#18 0x00007f44d2126850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

This is the same issue I've reported and attempting to solve. BTW when I use Oracle 12.2 I don't see this issue. Another ticket I've opened for Perl received a reply suggesting this is caused by an Oracle worker thread leaking into Perl to handle the SIGCHLD.

I just found yet another #179 ticket for the same issue.