perl5.10.0: caching environment causes panic in some modules (esp. Term::ReadLine::Gnu)

[p5pRT]

Migrated from rt.perl.org#56500 (status was 'open')

Searchable as RT56500$

[p5pRT]

From gawrilow@mail.math.tu-berlin.de

This is a bug report for perl from gawrilow@​math.tu-berlin.de,
generated with the help of perlbug 1.36 running under perl 5.10.0.

---

This bug has already been submitted by somebody else at rt.cpan.org,
ticket #37194, but without further analysis. The true reason for the failure
lies in perl core, not in the Term​::ReadLine​::Gnu module.

At the very beginning, perl makes a copy of the environment and sets the global
variable environ to point to it. Each environment string is allocated with perl's own
safesysmalloc.

Later on, the initialization routine of the GNU readline library rl_initialize
modifies the environment variables LINES and COLUMNS. Obviously, it does not know
anything about perl's memory management and uses the normal free/malloc.

During the global destruction perl tries to free the copies of the environment strings;
when it proceeds with the LINES or COLUMNS, it recognizes the lack of the hidden header
in front of the string and issues a panic() call.

In the documentation of the GNU readline library I didn't find any suggestions how to suppress
the environment modification; even if there were some possibility to do this, there are surely
a lot of other C/C++ libraries with perl wrappers, which might try to modify the environment.
On the other hand, no perl module has a chance to prevent environment caching, as the only flag
controlling this (PL_use_safe_putenv) is read off much earlier during the initial bootstrapping.

I'd suggest to give up the environment caching at all, or make it optional, on behalf and under
full responsibility of the user.

An alternative solution could be a pair of routines to be called from
each suspicious wrapper module, the first one restoring the original environment and the second one
checking it for updated strings and reallocating them using perl's memory management.
But it seems me very hard to guarantee that every module's author will really use it.

---

---

Flags​:
  category=core
  severity=high

---

Site configuration information for perl 5.10.0​:

Configured by gawrilow at Fri Jun 20 14​:26​:30 CEST 2008.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration​:
  Platform​:
  osname=linux, osvers=2.6.18.8-0.9-default, archname=x86_64-linux-debug-thread-multi
  uname='linux trillian 2.6.18.8-0.9-default #1 smp sun feb 10 22​:48​:05 utc 2008 x86_64 x86_64 x86_64 gnulinux '
  config_args=''
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -DDEBUG_LEAKING_SCALARS -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-g',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -DDEBUG_LEAKING_SCALARS'
  ccversion='', gccversion='4.1.2 20061115 (prerelease) (SUSE Linux)', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -L/usr/local/lib'
  libpth=/usr/local/lib /lib /usr/lib /lib64 /usr/lib64 /usr/local/lib64
  libs=-lnsl -lndbm -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
  libc=/lib/libc-2.5.so, so=so, useshrplib=true, libperl=libperl.so
  gnulibc_version='2.5'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/local/lib/perl5/5.10.0/x86_64-linux-debug-thread-multi/CORE'
  cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib'

Locally applied patches​:
 

---

@​INC for perl 5.10.0​:
  /usr/local/lib/perl5/5.10.0/x86_64-linux-debug-thread-multi
  /usr/local/lib/perl5/5.10.0
  /usr/local/lib/perl5/site_perl/5.10.0/x86_64-linux-debug-thread-multi
  /usr/local/lib/perl5/site_perl/5.10.0
  /usr/local/lib/perl5/site_perl/5.8.8
  /usr/local/lib/perl5/site_perl/5.8.7
  /usr/local/lib/perl5/site_perl
  .

---

Environment for perl 5.10.0​:
  HOME=/homes/combi/gawrilow
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/homes/combi/Software/x86_64​:/homes/combi/gawrilow/bin​:/bin​:/usr/bin​:/usr/X11R6/bin​:/opt/kde3/bin​:/opt/gnome/bin​:/usr/site-local/bin​:/usr/site-local32/bin​:/usr/site-local/java/bin​:/usr/local/bin​:/usr/site-local/share/bin​:/homes/combi/gawrilow/bin/icc-root/bin
  PERL_BADLANG (unset)
  SHELL=/bin/csh

[p5pRT]

From @jkeenan

On Tue Jul 01 04​:26​:25 2008, gawrilow@​mail.math.tu-berlin.de wrote​:

This bug has already been submitted by somebody else at rt.cpan.org,
ticket #37194, but without further analysis. The true reason for the
failure
lies in perl core, not in the Term​::ReadLine​::Gnu module.

Since https://rt.cpan.org/Public/Bug/Display.html?id=37194 has been
resolved by Term​::ReadLine​::Gnu's maintainer, I wonder whether there are
still outstanding issues in this RT.

At the very beginning, perl makes a copy of the environment and sets
the global
variable environ to point to it. Each environment string is allocated
with perl's own
safesysmalloc.

Later on, the initialization routine of the GNU readline library
rl_initialize
modifies the environment variables LINES and COLUMNS. Obviously, it
does not know
anything about perl's memory management and uses the normal
free/malloc.

During the global destruction perl tries to free the copies of the
environment strings;
when it proceeds with the LINES or COLUMNS, it recognizes the lack of
the hidden header
in front of the string and issues a panic() call.

In the documentation of the GNU readline library I didn't find any
suggestions how to suppress
the environment modification; even if there were some possibility to
do this, there are surely
a lot of other C/C++ libraries with perl wrappers, which might try to
modify the environment.
On the other hand, no perl module has a chance to prevent environment
caching, as the only flag
controlling this (PL_use_safe_putenv) is read off much earlier during
the initial bootstrapping.

I'd suggest to give up the environment caching at all, or make it
optional, on behalf and under
full responsibility of the user.

An alternative solution could be a pair of routines to be called from
each suspicious wrapper module, the first one restoring the original
environment and the second one
checking it for updated strings and reallocating them using perl's
memory management.
But it seems me very hard to guarantee that every module's author will
really use it.

Do people feel there is a real problem here? If so, would either of the
original poster's suggestions prove beneficial?

Thank you very much.
Jim Keenan

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @jkeenan

On Sat Dec 17 19​:17​:16 2011, jkeenan wrote​:

On Tue Jul 01 04​:26​:25 2008, gawrilow@​mail.math.tu-berlin.de wrote​:

This bug has already been submitted by somebody else at rt.cpan.org,
ticket #37194, but without further analysis. The true reason for the
failure
lies in perl core, not in the Term​::ReadLine​::Gnu module.

Since https://rt.cpan.org/Public/Bug/Display.html?id=37194 has been
resolved by Term​::ReadLine​::Gnu's maintainer, I wonder whether there are
still outstanding issues in this RT.

At the very beginning, perl makes a copy of the environment and sets
the global
variable environ to point to it. Each environment string is allocated
with perl's own
safesysmalloc.

Later on, the initialization routine of the GNU readline library
rl_initialize
modifies the environment variables LINES and COLUMNS. Obviously, it
does not know
anything about perl's memory management and uses the normal
free/malloc.

During the global destruction perl tries to free the copies of the
environment strings;
when it proceeds with the LINES or COLUMNS, it recognizes the lack of
the hidden header
in front of the string and issues a panic() call.

In the documentation of the GNU readline library I didn't find any
suggestions how to suppress
the environment modification; even if there were some possibility to
do this, there are surely
a lot of other C/C++ libraries with perl wrappers, which might try to
modify the environment.
On the other hand, no perl module has a chance to prevent environment
caching, as the only flag
controlling this (PL_use_safe_putenv) is read off much earlier during
the initial bootstrapping.

I'd suggest to give up the environment caching at all, or make it
optional, on behalf and under
full responsibility of the user.

An alternative solution could be a pair of routines to be called from
each suspicious wrapper module, the first one restoring the original
environment and the second one
checking it for updated strings and reallocating them using perl's
memory management.
But it seems me very hard to guarantee that every module's author will
really use it.

Do people feel there is a real problem here? If so, would either of the
original poster's suggestions prove beneficial?

There has been no response to these questions in the past 14 months.

I recommend keeping this ticket open for 4 more weeks and then closing
it if no one takes the discussion forward.

Thank you very much.
Jim Keenan

[p5pRT]

From @iabyn

On Sun, Feb 03, 2013 at 12​:06​:07PM -0800, James E Keenan via RT wrote​:

There has been no response to these questions in the past 14 months.

I recommend keeping this ticket open for 4 more weeks and then closing
it if no one takes the discussion forward.

At a quick glance I think that there may still be issues. I recommend
keeping the ticket open.

--
The optimist believes that he lives in the best of all possible worlds.
As does the pessimist.