exit()/exec() crashes Perl after fork

[p5pRT]

Migrated from rt.perl.org#21382 (status was 'open')

Searchable as RT21382$

[p5pRT]

From Jens.Lippmann@t-mobile.de

Created by jens.lippmann@t-mobile.de

Hello!

I'm into Perl for more than 8 years and I cannot overemphasize that I
appreciated every single minute programming with it.

Now, I actually spent quite a large amount of time to find a nasty bug
that crashes Perl on Windows NT when I call fork and exec.
The problem is not related to any specific Perl but rather sticks to Perl
ports that create threads instead of processes.
As soon as the exec'd task terminates there is a considerable high chance
(obviously a race condition) to crash perl.
When I replace exec() with system() and exit() the problem pertains, but
not if I use system() and POSIX​::_exit().

I am sending this bug report to the mainstream Perl development
instead of any specific Win32 support group such as ActiveState or
Indigo just because the 'bug' I am referring herein seems to be
related in a deficient Perl documentation on this topic.

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps
out of this remedy, and to extend the fork/exec sections in the
perlfuncs doc module, and to overhaul the notorious
'if ($pid = fork()) { ... } else { ... exec/exit; }'
examples that are used throughout the perlipc doc.

It is generally a bad idea to use exit() after fork(), unless you want to
create a daemon process (see http​://www.erlenstar.demon.co.uk/unix/faq_toc.html
"1.1.3 Why use_exit rather than exit in the child branch of a fork?").
It is also a bad idea to use exec() after fork() on a Win32 port for the simple
reason that exec() itself performs cleanup action on process termination
similar to exit().

It is no good way to leave the user undocumented and let the
developers run into the same obstacles again with every new release.
Meanwhile there are half a dozen Win32 module extensions on CPAN, but
none seems to come with support throughout all Win32 platforms or
availability for any perl distribution.
For the sake of transparency and portability, why can't Perl itself
solve the exit handling to smooth out thread and non-thread forks?
I recommend to either
- keep the transparency of fork() but then make exit() transparent as well
  (i.e. remember the 'thread' state and act accordingly), or
- do not support fork() on Win32 but vfork() and provide a function like
  vexit()

I further recommend to generally incorporate POSIX​::_exit() into the
Perl core and to add a hint about the batch command 'start' which can
be used to conveniently spawn a task under Win32.

kind regards
Jens Lippmann

Perl Info

Flags:
    category=docs
    severity=medium

Site configuration information for perl v5.6.1:

Configured by Indy at Sun May  6 21:35:42 2001.

Summary of my perl5 (revision 5 version 6 subversion 1) configuration:
  Platform:
    osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    usethreads=undef use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=undef d_sfio=undef uselargefiles=undef usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
  Compiler:
    cc='cl', ccflags ='-nologo -O1 -MD -DNDEBUG -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT  -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DPERL_MSVCRT_READFIX',
    optimize='-O1 -MD -DNDEBUG',
    cppflags='-DWIN32'
    ccversion='', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=4
    alignbytes=8, usemymalloc=n, prototype=define
  Linker and Libraries:
    ld='', ldflags ='-nologo -nodefaultlib -release  -libpath:"f:\dev\dp\perl-5.6_install\lib\CORE"  -machine:x86'
    libpth=\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl56.lib
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release  -libpath:"f:\dev\dp\perl-5.6_install\lib\CORE"  -machine:x86'

Locally applied patches:
    ACTIVEPERL_LOCAL_PATCHES_ENTRY


@INC for perl v5.6.1:
    C:/PROGS/perl/lib
    C:/PROGS/perl/site/lib
    .


Environment for perl v5.6.1:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=c:\Progs\Rational\ClearCase\bin;W:\TOOLS\CLEARMOBIL\bin;C:\WINNT\system32;C:\WINNT;C:\PROGS\TEAMLINK;C:\PROGS\Rational\COMMON;C:\PROGS\VS\COMMON\Tools\WinNT;C:\PROGS\VS\COMMON\MSDev\Bin;C:\PROGS\VS\COMMON\Tools;c:\progs\vs\vc\bin;C:\PROGS\MAESTRO;;C:\PROGS\Office\Office;E:\rtools;O:\wna_source\wna_apps\src\regtest\tools;O:\wna_source\wna_apps\src\helper\batch;O:\wna_source\wna_apps\src\onc-rpc-nt\bin;R:\oracle\816\bin;O:\tps_source\cyg\src\cygwin\bin;O:\private_binary\bin\nt\Debug;O:\private_binary\bin\nt\Release;O:\car_binary\bin\nt\debug;O:\car_binary\bin\nt\release;O:\tps_binary\bin\nt\debug;O:\tps_binary\bin\nt\release;O:\tfs_binary\bin\nt\debug;O:\tfs_binary\bin\nt\release;L:\Carmen.ABN\5.0\common;C:\progs\xmlpad;C:\progs\textpad;C:\progs\perl\bin
    PERL_BADLANG (unset)
    SHELL (unset)

[p5pRT]

arthur@contiller.se - Status changed from 'new' to 'open'

[p5pRT]

From @cwest

[Jens.Lippmann <!--c--> <i>at</i> <!--a--> t-mobile.de - Wed Feb
26 13​:26​:38 2003]​:

This is a bug report for perl from jens.lippmann <!--c--> <i>at</i> <!--a--> t-
mobile.de,
generated with the help of perlbug 1.33 running under perl v5.6.1.

-----------------------------------------------------------------
[Please enter your report here]
Hello!

I'm into Perl for more than 8 years and I cannot overemphasize that I
appreciated every single minute programming with it.

Excellent!

Now, I actually spent quite a large amount of time to find a nasty bug
that crashes Perl on Windows NT when I call fork and exec.
The problem is not related to any specific Perl but rather sticks to
Perl
ports that create threads instead of processes.
As soon as the exec'd task terminates there is a considerable high
chance
(obviously a race condition) to crash perl.
When I replace exec() with system() and exit() the problem pertains,
but
not if I use system() and POSIX​::_exit().

I am sending this bug report to the mainstream Perl development
instead of any specific Win32 support group such as ActiveState or
Indigo just because the 'bug' I am referring herein seems to be
related in a deficient Perl documentation on this topic.

Sounds reasonable...

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps
out of this remedy, and to extend the fork/exec sections in the
perlfuncs doc module, and to overhaul the notorious
'if ($pid = fork()) { ... } else { ... exec/exit; }'
examples that are used throughout the perlipc doc.

How's this coming along?

It is generally a bad idea to use exit() after fork(), unless you want
to
create a daemon process (see
http​://www.erlenstar.demon.co.uk/unix/faq_toc.html
"1.1.3 Why use_exit rather than exit in the child branch of a
fork?").
It is also a bad idea to use exec() after fork() on a Win32 port for
the simple
reason that exec() itself performs cleanup action on process
termination
similar to exit().

It is no good way to leave the user undocumented and let the
developers run into the same obstacles again with every new release.
Meanwhile there are half a dozen Win32 module extensions on CPAN, but
none seems to come with support throughout all Win32 platforms or
availability for any perl distribution.
For the sake of transparency and portability, why can't Perl itself
solve the exit handling to smooth out thread and non-thread forks?
I recommend to either
- keep the transparency of fork() but then make exit() transparent as
well
(i.e. remember the 'thread' state and act accordingly), or
- do not support fork() on Win32 but vfork() and provide a function
like
vexit()

I further recommend to generally incorporate POSIX​::_exit() into the
Perl core and to add a hint about the batch command 'start' which can
be used to conveniently spawn a task under Win32.

This isn't quite a documentation bug, so lets work on the documentation bug first.
Please let me know where you are on this, I'd like to get this bug resolved. Thanks.

  Casey West

[p5pRT]

From @cwest

[replying and posting to p5p]

[Jens.Lippmann <!--c--> <i>at</i> <!--a--> t-mobile.de - Wed Feb 26 13​:26​:38 2003]​:

Now, I actually spent quite a large amount of time to find a nasty bug
that crashes Perl on Windows NT when I call fork and exec.
The problem is not related to any specific Perl but rather sticks to
Perl
ports that create threads instead of processes.
As soon as the exec'd task terminates there is a considerable high
chance
(obviously a race condition) to crash perl.
When I replace exec() with system() and exit() the problem pertains,
but
not if I use system() and POSIX​::_exit().

Ok, I have no access to a Win32 machine so I can't go on testing
this, but I can believe you.

I am sending this bug report to the mainstream Perl development
instead of any specific Win32 support group such as ActiveState or
Indigo just because the 'bug' I am referring herein seems to be
related in a deficient Perl documentation on this topic.

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps
out of this remedy, and to extend the fork/exec sections in the
perlfuncs doc module, and to overhaul the notorious
'if ($pid = fork()) { ... } else { ... exec/exit; }'
examples that are used throughout the perlipc doc.

None of this has been done yet. Is it all that necessary? Has anyone
else found the need to have this documented in some way, somewhere?

[p5pRT]

From Jens.Lippmann@t-mobile.de

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps
out of this remedy, and to extend the fork/exec sections in the
perlfuncs doc module, and to overhaul the notorious
'if ($pid = fork()) { ... } else { ... exec/exit; }'
examples that are used throughout the perlipc doc.

None of this has been done yet. Is it all that necessary? Has anyone
else found the need to have this documented in some way, somewhere?

I'm currently building up an extension to perlfaq8.pod
top 'How do I start a process in the background?'.

Later I'll try to narrow down my script in size to a
sniplet that is able to reproduce the crash.
Although it might be troublesome since the crash (race condition)
seems to be related to the size of the process image.

kind regards
Jens

[p5pRT]

From sky@nanisky.com

On torsdag, maj 15, 2003, at 18​:50 Europe/Stockholm, Lippmann Jens,E2
wrote​:

Later I'll try to narrow down my script in size to a
sniplet that is able to reproduce the crash.
Although it might be troublesome since the crash (race condition)
seems to be related to the size of the process image.

kind regards
Jens

I would think it is more likely related to a module you are using that
doesn't cope well with the psuedo fork.

Can we have a backtrace?

Arthur

[p5pRT]

From Jens.Lippmann@t-mobile.de

Hi

I cannot shrink the script, as soon as I remove a certain quantity
of text the crash disappears.
Well I admit that I use file IO quite extensively and keep file handles
in memory, so that both parent and child refer to the same descriptors.
BUT the child does not make use of those descriptors, it simply calls
'system("echo test") && exit'
Immediately after the child exits Perl will crash.

I'm in an urgent need of a Perl Debug-Build because I can't afford the
time to play catch-me with this bug.
Any Perl that runs on Win32 and uses threads is welcome.
My crash occurs with Indigo 5.6.1/5.8.0 as well as ActiveState 5.6.1.
If someone can point me to a Win32 Perl development distribution I
would be *greatly* relieved.

-----Ursprüngliche Nachricht-----
Von​: Lippmann Jens,E2
Gesendet am​: Donnerstag, 15. Mai 2003 18​:50
An​: 'perlbug-followup@​perl.org'
Betreff​: AW​: [perl #21382] exit()/exec() crashes Perl after fork

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps
out of this remedy, and to extend the fork/exec sections in the
perlfuncs doc module, and to overhaul the notorious
'if ($pid = fork()) { ... } else { ... exec/exit; }'
examples that are used throughout the perlipc doc.

None of this has been done yet. Is it all that necessary? Has anyone
else found the need to have this documented in some way, somewhere?

I'm currently building up an extension to perlfaq8.pod
top 'How do I start a process in the background?'.

Later I'll try to narrow down my script in size to a
sniplet that is able to reproduce the crash.
Although it might be troublesome since the crash (race condition)
seems to be related to the size of the process image.

kind regards
Jens

[p5pRT]

From @bulk88

To move this old ticket along

On Sat May 24 00​:40​:29 2003, Jens.Lippmann@​t-mobile.de wrote​:

Hi

I cannot shrink the script, as soon as I remove a certain quantity
of text the crash disappears.
Well I admit that I use file IO quite extensively and keep file handles
in memory, so that both parent and child refer to the same descriptors.
BUT the child does not make use of those descriptors, it simply calls
'system("echo test") && exit'
Immediately after the child exits Perl will crash.

I'm in an urgent need of a Perl Debug-Build because I can't afford the
time to play catch-me with this bug.
Any Perl that runs on Win32 and uses threads is welcome.
My crash occurs with Indigo 5.6.1/5.8.0 as well as ActiveState 5.6.1.
If someone can point me to a Win32 Perl development distribution I
would be *greatly* relieved.

-----Ursprüngliche Nachricht-----
Von​: Lippmann Jens,E2
Gesendet am​: Donnerstag, 15. Mai 2003 18​:50
An​: 'perlbug-followup@​perl.org'
Betreff​: AW​: [perl #21382] exit()/exec() crashes Perl after fork

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps
out of this remedy, and to extend the fork/exec sections in the
perlfuncs doc module, and to overhaul the notorious
'if ($pid = fork()) { ... } else { ... exec/exit; }'
examples that are used throughout the perlipc doc.

None of this has been done yet. Is it all that necessary? Has anyone
else found the need to have this documented in some way, somewhere?

I'm currently building up an extension to perlfaq8.pod
top 'How do I start a process in the background?'.

I dont think this patch will ever come since it is 2013. The text, in question, now has a disclaimer that fork/exec aren't the only way to background run.

---

=head2 How do I start a process in the background?

(contributed by brian d foy)

There's not a single way to run code in the background so you don't
have to wait for it to finish before your program moves on to other
tasks. Process management depends on your particular operating system,
and many of the techniques are in L<perlipc>.

Several CPAN modules may be able to help, including C<IPC​::Open2> or
C<IPC​::Open3>, C<IPC​::Run>, C<Parallel​::Jobs>,
C<Parallel​::ForkManager>, C<POE>, C<Proc​::Background>, and
C<Win32​::Process>. There are many other modules you might use, so
check those namespaces for other options too.

---

Later I'll try to narrow down my script in size to a
sniplet that is able to reproduce the crash.
Although it might be troublesome since the crash (race condition)
seems to be related to the size of the process image.

Is this a crash with 5.005 threads?

--
bulk88 ~ bulk88 at hotmail.com

[toddr]

@bulk88 the original report was on 5.6.1