foreach spuriously autovivifies

[p5pRT]

Migrated from rt.perl.org#2166 (status was 'open')

Searchable as RT2166$

[p5pRT]

From mjtg@cus.cam.ac.uk

  foreach (@​h{a, b}) { }

autovivifies $h{a} and $h{b}.

Yes, I understand why it happens.

Mike Guy

% perl -V
Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration​:
  Platform​:
  osname=sunos, osvers=4.1.3, archname=sun4-sunos
  uname=''
  hint=previous, useposix=true, d_sigaction=define
  usethreads=undef useperlio=undef d_sfio=undef
  Compiler​:
  cc='gcc', optimize='-O', gccversion=2.7.2.3
  cppflags='-I/usr/local/include -DREG_INFTY=22790'
  ccflags ='-I/usr/local/include -DREG_INFTY=22790'
  stdchar='unsigned char', d_stdstdio=define, usevfork=true
  intsize=4, longsize=4, ptrsize=4, doublesize=8
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8
  alignbytes=8, usemymalloc=y, prototype=define
  Linker and Libraries​:
  ld='ld', ldflags =' -L/usr/local/lib'
  libpth=/usr/local/lib /lib /usr/lib /usr/ucblib
  libs=-ldbm -ldl -lm -lc -lposix
  libc=/lib/libc.so.1.8, so=so, useshrplib=false, libperl=libperl.a
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
  cccdlflags='-fpic', lddlflags='-assert nodefinitions -L/usr/local/lib'

Characteristics of this binary (from libperl)​:
  Locally applied patches​:
  pos.list
  IO.file.return
  int.delimiter
  bigfloat
  Built under sunos
  Compiled at Jul 7 1999 16​:09​:54
  @​INC​:
  /home/mjtg/perl5.005_03/lib
  /home/mjtg/perl5.005_03/lib
  /home/mjtg/perl5.005_03/lib
  .

[p5pRT]

From @schwern

[mjtg <!--c--> <i>at</i> <!--a--> cus.cam.ac.uk - Thu Feb 17 18​:57​:54 2000]​:

       foreach \(@​h\{a\, b\}\) \{ \}

autovivifies $h{a} and $h{b}.

Issue still exists in 5.8.1 RC 2

[p5pRT]

From module@renee-baecker.de

Is it really a bug? I would expect these elements to be autovivified...

[p5pRT]

From @smpeters

On Sun, Jun 15, 2008 at 6​:02 AM, reneeb via RT
<perlbug-followup@​perl.org> wrote​:

Is it really a bug? I would expect these elements to be autovivified...

Unfortunately, it has pretty much agreed that it is a bug, but due to
the regular use of this bug as a feature, it cannot be fixed. See
http​://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2006-02/msg00823.html
for additional examples of this bug.

Steve Peters
steve@​fisharerojo.org

[p5pRT]

From @davidnicol

On 6/20/08, Steve Peters <steve@​fisharerojo.org> wrote​:

Unfortunately, it has pretty much agreed that it is a bug, but due to

but not universally. The opposing POV holds that the non-autoviv of
nonexistent hash elements used as subroutine arguments was misguided
due to the fact that it greatly confuses the issues and creates false
expectations and is therefore the aspect (more general term used to
avoid either "feature" or "bug") which should be deprecated and backed
out, reaplced with instructions to doublequote your hash element
look-ups if you want to pass them to subroutines without having them
autovivified.

  my %empty;
  some_function($empty{hooha}); # once autovivved, doesn't now, until
$_[0] assigned to
  some_function("$empty{hooha}"); # never autovivved, still doesn't

[p5pRT]

From @iabyn

On Fri, Jun 20, 2008 at 01​:03​:31PM -0500, David Nicol wrote​:

reaplced with instructions to doublequote your hash element
look-ups if you want to pass them to subroutines without having them
autovivified.

my %empty;
some_function($empty{hooha}); # once autovivved, doesn't now, until
$_[0] assigned to
some_function("$empty{hooha}"); # never autovivved, still doesn't

Which would be pretty unfortunate if $empty{hooha} was a ref.

--
"Procrastination grows to fill the available time"
  -- Mitchell's corollary to Parkinson's Law

[p5pRT]

From blgl@hagernas.com

In article <20080622132012.GM12186@​iabyn.com>, davem@​iabyn.com (Dave Mitchell)
wrote​:

On Fri, Jun 20, 2008 at 01​:03​:31PM -0500, David Nicol wrote​:

reaplced with instructions to doublequote your hash element
look-ups if you want to pass them to subroutines without having them
autovivified.

my %empty;
some_function($empty{hooha}); # once autovivved, doesn't now, until
$_[0] assigned to
some_function("$empty{hooha}"); # never autovivved, still doesn't

Which would be pretty unfortunate if $empty{hooha} was a ref.

Which is why you use do-BLOCK instead​:

  some_function(do { $empty{hooha}; });
  # never autovivifies, preserves references

I don't think this behaviour is documented anywhere, though.

/Bo Lindbergh

[p5pRT]

From @ysth

On Sun, June 22, 2008 12​:25 pm, Bo Lindbergh wrote​:

In article <20080622132012.GM12186@​iabyn.com>, davem@​iabyn.com (Dave
Mitchell) wrote​:

On Fri, Jun 20, 2008 at 01​:03​:31PM -0500, David Nicol wrote​:

reaplced with instructions to doublequote your hash element look-ups
if you want to pass them to subroutines without having them
autovivified.

my %empty; some_function($empty{hooha}); # once autovivved, doesn't
now, until $_[0] assigned to
some_function("$empty{hooha}"); # never autovivved, still doesn't

Which would be pretty unfortunate if $empty{hooha} was a ref.

Which is why you use do-BLOCK instead​:

some_function(do { $empty{hooha}; }); # never autovivifies, preserves
references

Or some_function(scalar $empty{hooha})

[p5pRT]

From jjnoakes@gmail.com

Created by jjnoakes@gmail.com

  This code​:

  my %a;
  die "Bad 1\n" if exists $a{'FOO'};
  foreach my $b ( $a{'FOO'}, $a{'BAR'} ) {}
  die "Bad 2\n" if exists $a{'FOO'};

  will die with "Bad 2". I don't believe the aliasing aspect
  of the "foreach" loop should actually autovivify hash elements...

  This is particularly bad for tied hashes​:

  foreach my $x ( $ENV{'TMPDIR'}, $ENV{'TMP'}, ... ) {}

  That will actually cause environment variables to spring to life.
  Aliasing should not have this side-effect.

  It is not good enough to create the element and delete it later;
  no element should be created at any point in this code.

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl v5.8.8:

Configured by Mandriva at Fri May  9 03:51:40 EDT 2008.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.24.4-server-1mnb, archname=i386-linux
    uname='linux titan.mandriva.com 2.6.24.4-server-1mnb #1 smp thu mar 27
15:24:36 cet 2008 i686 intel(r) xeon(r) cpu x3210 @ 2.13ghz gnulinux '
    config_args='-des -Dinc_version_list=5.8.7 5.8.7/i386-linux 5.8.6
5.8.6/i386-linux 5.8.5 5.8.4 5.8.3 5.8.2 5.8.1 5.8.0 5.6.1 5.6.0
-Darchname=i386-linux -Dcc=gcc -Doptim$
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-fno-strict-aliasing -pipe
-Wdeclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-O2 -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fomit-frame-pointer -march=i586 -mtune=generic
-fasynchronous-unwind-tables',
    cppflags='-fno-strict-aliasing -pipe -Wdeclaration-after-statement
-I/usr/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='4.1.2 20070302 (prerelease)
(4.1.2-1mdv2007.1)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lndbm -lgdbm -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=/lib/libc-2.4.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.4'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E
-Wl,-rpath,/usr/lib/perl5/5.8.8/i386-linux/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    Mandriva Linux patches


@INC for perl v5.8.8:
    /usr/lib/perl5/5.8.8/i386-linux
    /usr/lib/perl5/5.8.8
    /usr/lib/perl5/site_perl/5.8.8/i386-linux
    /usr/lib/perl5/site_perl/5.8.8
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.8.8/i386-linux
    /usr/lib/perl5/vendor_perl/5.8.8
    /usr/lib/perl5/vendor_perl/5.8.7
    /usr/lib/perl5/vendor_perl/5.8.7/i386-linux
    /usr/lib/perl5/vendor_perl/5.8.6
    /usr/lib/perl5/vendor_perl
    .


Environment for perl v5.8.8:
    HOME=/home/jj
    LANG=C
    LANGUAGE=en_US:en
    LC_ADDRESS=C
    LC_COLLATE=C
    LC_CTYPE=C
    LC_IDENTIFICATION=C
    LC_MEASUREMENT=C
    LC_MESSAGES=C
    LC_MONETARY=C
    LC_NAME=C
    LC_NUMERIC=C
    LC_PAPER=C
    LC_SOURCED=1
    LC_TELEPHONE=C
    LC_TIME=C
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/bin:/bin:/usr/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From @demerphq

2009/3/26 Jason Noakes <perlbug-followup@​perl.org>​:

# New Ticket Created by  Jason Noakes
# Please include the string​:  [perl #64228]
# in the subject line of all future correspondence about this issue.
# <URL​: http​://rt.perl.org/rt3/Ticket/Display.html?id=64228 >

This is a bug report for perl from jjnoakes@​gmail.com,
generated with the help of perlbug 1.35 running under perl v5.8.8.

-----------------------------------------------------------------
[Please enter your report here]

 This code​:

   my %a;
   die "Bad 1\n" if exists $a{'FOO'};
   foreach my $b ( $a{'FOO'}, $a{'BAR'} ) {}
   die "Bad 2\n" if exists $a{'FOO'};

 will die with "Bad 2". I don't believe the aliasing aspect
 of the "foreach" loop should actually autovivify hash elements...

 This is particularly bad for tied hashes​:

   foreach my $x ( $ENV{'TMPDIR'}, $ENV{'TMP'}, ... ) {}

 That will actually cause environment variables to spring to life.
 Aliasing should not have this side-effect.

 It is not good enough to create the element and delete it later;
 no element should be created at any point in this code.

No, thats wrong. Aliasing SHOULD have this effect.

Or else this wouldn't work​:

perl -e'my %x=(a=>1); $_='muhahaha' for $x{b}; use Data​::Dumper;
print Dumper(\%x)'
$VAR1 = {
  'a' => 1,
  'b' => 'muhahaha'
  };

Its irritating i agree, but the list provided to foreach() is in
LVALUE context because of the aliasing, and thus it autovivifies the
slot.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @davidnicol

On Fri, Mar 27, 2009 at 12​:23 PM, demerphq <demerphq@​gmail.com> wrote​:

No, thats wrong. Aliasing SHOULD have this effect.

Or else this wouldn't work​:

 perl -e'my %x=(a=>1); $_='muhahaha' for $x{b}; use Data​::Dumper;
print Dumper(\%x)'
$VAR1 = {
         'a' => 1,
         'b' => 'muhahaha'
       };

Its irritating i agree, but the list provided to foreach() is in
LVALUE context because of the aliasing, and thus it autovivifies the
slot.

Yves

Yet, in Mr. Noakes' defense,

perl -e'my %x=(a=>1); sub xy {$_[1]="muhahaha" }; xy $x{b},
$x{bwahaha}; use Data​::Dumper; print Dumper(\%x)'
$VAR1 = {
  'bwahaha' => 'muhahaha',
  'a' => 1
  };

does not autoviv $x{b}, demonstrating non-autivivving aliasing and
creating a reasonable expectation that for aliases won't autoviv
except when actually used as lvalues, just like arg aliases don't.

[p5pRT]

From @smpeters

On Fri, Mar 27, 2009 at 1​:52 PM, David Nicol <davidnicol@​gmail.com> wrote​:

On Fri, Mar 27, 2009 at 12​:23 PM, demerphq <demerphq@​gmail.com> wrote​:

No, thats wrong. Aliasing SHOULD have this effect.

Or else this wouldn't work​:

 perl -e'my %x=(a=>1); $_='muhahaha' for $x{b}; use Data​::Dumper;
print Dumper(\%x)'
$VAR1 = {
         'a' => 1,
         'b' => 'muhahaha'
       };

Its irritating i agree, but the list provided to foreach() is in
LVALUE context because of the aliasing, and thus it autovivifies the
slot.

Yves

Yet, in Mr. Noakes' defense,

 perl -e'my %x=(a=>1); sub xy {$_[1]="muhahaha" }; xy $x{b},
$x{bwahaha}; use Data​::Dumper; print Dumper(\%x)'
$VAR1 = {
         'bwahaha' => 'muhahaha',
         'a' => 1
       };

does not autoviv $x{b}, demonstrating non-autivivving aliasing and
creating a reasonable expectation that for aliases won't autoviv
except when actually used as lvalues, just like arg aliases don't.

This appears to be another case of "RT #2166​: foreach spuriously
autovivifies". It is widely considered to be a bug, but is also
regularly used as a feature. It is well documented, but is unlikely
to ever be fixed due to its extensive use.

Steve Peters
steve@​fisharerojo.org

[p5pRT]

From @gbarr

On Mar 27, 2009, at 1​:52 PM, David Nicol wrote​:

On Fri, Mar 27, 2009 at 12​:23 PM, demerphq <demerphq@​gmail.com> wrote​:

No, thats wrong. Aliasing SHOULD have this effect.

Or else this wouldn't work​:

perl -e'my %x=(a=>1); $_='muhahaha' for $x{b}; use Data​::Dumper;
print Dumper(\%x)'
$VAR1 = {
'a' => 1,
'b' => 'muhahaha'
};

Its irritating i agree, but the list provided to foreach() is in
LVALUE context because of the aliasing, and thus it autovivifies the
slot.

Yves

Yet, in Mr. Noakes' defense,

perl -e'my %x=(a=>1); sub xy {$_[1]="muhahaha" }; xy $x{b},
$x{bwahaha}; use Data​::Dumper; print Dumper(\%x)'
$VAR1 = {
'bwahaha' => 'muhahaha',
'a' => 1
};

does not autoviv $x{b}, demonstrating non-autivivving aliasing and
creating a reasonable expectation that for aliases won't autoviv
except when actually used as lvalues, just like arg aliases don't.

subs did at one time also autoviv but it was changed during 5.005
development (I think)

There are many places that auto-viv due to being in LVALUE (map and
grep for
example), but sub args were special cased out (I cannot remember why)

IMO, the fact that subs do not autoviv is the bug as they are the
exception

In the past I have used reverse to avoid the autoviv as it does not
place its
arguments in an LVALUE context

Graham.

[p5pRT]

From @rgs

2009/3/31 Graham Barr <gbarr@​pobox.com>​:

On Mar 27, 2009, at 1​:52 PM, David Nicol wrote​:

On Fri, Mar 27, 2009 at 12​:23 PM, demerphq <demerphq@​gmail.com> wrote​:

No, thats wrong. Aliasing SHOULD have this effect.

Or else this wouldn't work​:

 perl -e'my %x=(a=>1); $_='muhahaha' for $x{b}; use Data​::Dumper;
print Dumper(\%x)'
$VAR1 = {
        'a' => 1,
        'b' => 'muhahaha'
      };

Its irritating i agree, but the list provided to foreach() is in
LVALUE context because of the aliasing, and thus it autovivifies the
slot.

Yves

Yet, in Mr. Noakes' defense,

perl -e'my %x=(a=>1); sub xy {$_[1]="muhahaha" }; xy $x{b},
$x{bwahaha}; use Data​::Dumper; print Dumper(\%x)'
$VAR1 = {
        'bwahaha' => 'muhahaha',
        'a' => 1
      };

does not autoviv $x{b}, demonstrating non-autivivving aliasing and
creating a reasonable expectation that for aliases won't autoviv
except when actually used as lvalues, just like arg aliases don't.

subs did at one time also autoviv but it was changed during 5.005
development (I think)

There are many places that auto-viv due to being in LVALUE (map and grep for
example), but sub args were special cased out (I cannot remember why)

IMO, the fact that subs do not autoviv is the bug as they are the exception

In the past I have used reverse to avoid the autoviv as it does not place
its
arguments in an LVALUE context

On that subject, and work to be done, see
http​://consttype.blogspot.com/2009/03/autovivification-in-perl-5.html
(blogged to -maybe- get a wider audience)