With long chains of references both Storable and Data::Dumper cause perl to fail

[p5pRT]

Migrated from rt.perl.org#36427 (status was 'open')

Searchable as RT36427$

[p5pRT]

From kynn@panix.com

I first observed this behavior when attempting to save a directed
acyclic graph (DAG) consisting of of about 5,000 nodes, but below I
give a minimalistic example that exhibits the same behavior.

The following script creates a long sequence of nested references.
The command-line argument to the script should be a positive integer,
corresponding to the length of this sequence of references.

With an argument greater than some system-dependent critical size, the
script causes perl to segfault​:

use strict;
use warnings;

my $size = shift;

my $x = [];
$x = [ $x ] for 1..$size;

require Storable;
Storable​::store( \$x, '/tmp/$0.$$' ) or die;

warn "ok\n";

__END__

For this script, in the system described in this report, the critical
value of the argument is 30826. With any argument below this number,
the script terminates normally; with an argument greater than or equal
to it, perl segfaults.

Similarly, if the two lines indicated by the "# *" in the script above
are replaced by

require Data​::Dumper;
Data​::Dumper->Dump( [ \$x ] ) or die;

the resulting script fails for any argument greater than or equal to
15406.

Two important differences between the two versions of the script is
that the one using Storable fails after the 'warn "ok\n"' line, and
always terminates quickly (whether successfully or not), even for
values of the input close to the critical value. In contrast, for the
version using Data​::Dumper, when the script fails the 'warn "ok\n"'
line never executes, and the time to success or failure grows sharply
as the value of the input approaches the critical value. But with the
DAG object that originally triggered the segfaulting, I observed this
pronounced slowing down near the critical input size with both
Data​::Dumper and Storable.

Most likely the reason for the segfaulting is that a sufficiently long
chain of links will cause the serialization process to start
thrashing. Simply breaking the chain of links in two unconnected
halves allows the serialization to succeed, even though the total
number of links has been reduced by only 1.

Perl Info

Flags:
    category=library
    severity=medium

Site configuration information for perl v5.8.6:

Configured by kynn at Tue Apr 12 12:24:11 EDT 2005.

Summary of my perl5 (revision 5 version 8 subversion 6) configuration:
  Platform:
    osname=linux, osvers=2.4.18-686-smp, archname=i686-linux-ld
    uname='linux luna 2.4.18-686-smp #1 smp sun apr 14 12:07:19 est 2002 i686 unknown '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=define
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='2.95.4 20011002 (Debian prerelease)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='long double', nvsize=12, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldbm -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=/lib/libc-2.2.5.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.2.5'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.6:
    /home/kynn/local/lib/perl/my
    /home/kynn/local/lib/perl/5.8.4
    /home/kynn/local/lib/perl/5.6.1
    /home/kynn/local/lib/perl5/5.8.6/i686-linux-ld
    /home/kynn/local/lib/perl5/5.8.6
    /home/kynn/local/lib/perl5/i686-linux-ld
    /home/kynn/local/lib/perl5
    /home/kynn/local/lib/perl5/5.8.6/i686-linux-ld
    /home/kynn/local/lib/perl5/5.8.6
    /home/kynn/local/lib/perl5/site_perl/5.8.6/i686-linux-ld
    /home/kynn/local/lib/perl5/site_perl/5.8.6
    /home/kynn/local/lib/perl5/site_perl
    .


Environment for perl v5.8.6:
    HOME=/home/kynn
    LANG=C
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/home/kynn/local/lib:/opt/java/jdk/lib/i386
    LOGDIR (unset)
    PATH=.:/home/kynn/local/bin:/home/kynn/local/usr/bin:/home/kynn/local/local/bin:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/home/kynn/java/javacc/bin:/opt/java/jdk/bin:/usr/pbs/bin:/home/kynn/opt/ant/bin
    PERL5LIB=/home/kynn/local/lib/perl/my:/home/kynn/local/lib/perl/5.8.4:/home/kynn/local/lib/perl/5.6.1:/home/kynn/local/lib/perl5
    PERL5_CPANPLUS_CONFIG=/home/kynn/.cpanplus/config
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

[p5pRT]

From @iabyn

On Wed, Jun 29, 2005 at 09​:46​:33PM -0000, kynn jones wrote​:

With an argument greater than some system-dependent critical size, the
script causes perl to segfault​:

use strict;
use warnings;

my $size = shift;

my $x = [];
$x = [ $x ] for 1..$size;

require Storable;
Storable​::store( \$x, '/tmp/$0.$$' ) or die;

It's the stack being trashed due to excessive recursion. No real way to
fix it short of rewriting Storable to be iterative rather than recursive.

The workaround is to increase the stack size available to the process
with 'uname -s'

--
In defeat, indomitable; in victory, insufferable
  -- Churchill on Montgomery

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'