IO::Handle::error doesn't report errors

[p5pRT]

Migrated from rt.perl.org#24067 (status was 'open')

Searchable as RT24067$

[p5pRT]

From peff-perlbug@peff.net

This is a bug report for perl from peff-perlbug@​peff.net,
generated with the help of perlbug 1.34 running under perl v5.8.0.

---

The following perl program writes 4096 bytes to a file, checking for errors​:

$ cat foo.pl
use IO​::File;
my $fh = new IO​::File($ARGV[0], 'w');
$fh->print('a' x 4096) || print "print failed​: $!\n";
print "errors​: " . ($fh->error() ? "yes ($!)" : "no") . "\n";

When running it on a regular file, it works as expected​:
$ perl foo.pl foo.out
errors​: no

However, when an error actually occurs, the print statement returns an error
but $fh->error does not return true​:
$ perl foo.pl /dev/full
print failed​: No space left on device
errors​: no

IO.xs shows that IO​::Handle​::error just calls PerlIO_error, which typically
just calls PerlIOBase_error to check the PERLIO_F_ERROR flag. However, if I
run gdb on the perl binary, I see that​:

- PerlIO_write is called with (in my case) f=0x81a70e4
- PerlIOBuf_flush correctly sets the flag with the same f=0x81a70e4, and
  sets the PERLIO_F_ERROR flag on f->flags
- PerlIO_error gets called with f=0x81a70e4 (from Perl_do_print), and
  correctly returns that an error occurred
- PerlIO_error gets called with f=0x81a70e0, whose flags don't have an
  error; this is *not* for the error message printed to stdout (I also saw
  that happen, and it has a different address).

Whatever the address of the original PerlIO handle is, the final call to
PerlIO_error always has an address 4 bytes less (the size of a pointer on my
platform).

I haven't been able to follow it past here since I'm unclear on exactly where
that pointer comes from (a backtrace shows it comes from XS_IO__Handle_error,
but I can't trace into that).

---

---

Flags​:
  category=core
  severity=low

---

Site configuration information for perl v5.8.0​:

Configured by Debian Project at Thu Sep 11 20​:57​:21 EST 2003.

Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration​:
  Platform​:
  osname=linux, osvers=2.4.21-xfs+ti1211, archname=i386-linux-thread-multi
  uname='linux kosh 2.4.21-xfs+ti1211 #1 sat jul 12 10​:35​:04 est 2003 i686 gnulinux '
  config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8.0 -Darchlib=/usr/lib/perl/5.8.0 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.0 -Dsitearch=/usr/local/lib/perl/5.8.0 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.0 -Dd_dosuid -des'
  hint=recommended, useposix=true, d_sigaction=define
  usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
  useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
  use64bitint=undef use64bitall=undef uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O3',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing'
  ccversion='', gccversion='3.3.2 20030908 (Debian prerelease)', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=4, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -L/usr/local/lib'
  libpth=/usr/local/lib /lib /usr/lib
  libs=-lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt
  perllibs=-ldl -lm -lpthread -lc -lcrypt
  libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libperl.so.5.8.0
  gnulibc_version='2.3.2'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
  cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches​:
 

---

@​INC for perl v5.8.0​:
  /home/peff/local/share/perl/5.8.0
  /home/peff/local/share/perl
  /home/peff/local/lib/perl5/site_perl
  /home/peff/local/lib/perl/5.8.0
  /home/peff/local/lib/perl
  /etc/perl
  /usr/local/lib/perl/5.8.0
  /usr/local/share/perl/5.8.0
  /usr/lib/perl5
  /usr/share/perl5
  /usr/lib/perl/5.8.0
  /usr/share/perl/5.8.0
  /usr/local/lib/site_perl
  .

---

Environment for perl v5.8.0​:
  HOME=/home/peff
  LANG=C
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/home/peff/local/bin​:/usr/local/bin​:/usr/bin​:/bin​:/usr/bin/X11​:/usr/games​:/sbin​:/usr/sbin​:/home/peff/fenris/build/bin​:/home/peff/work/mmake/bin​:/home/peff/work/sadm/bin
  PERL5LIB=/home/peff/local/share/perl​:/home/peff/local/lib/perl5/site_perl​:/home/peff/local/lib/perl
  PERL_BADLANG (unset)
  SHELL=/bin/bash

[p5pRT]

From @iabyn

On Mon, Sep 29, 2003 at 03​:43​:41PM -0000, peff-perlbug@​peff.net (via RT) wrote​:

-----------------------------------------------------------------
The following perl program writes 4096 bytes to a file, checking for errors​:

$ cat foo.pl
use IO​::File;
my $fh = new IO​::File($ARGV[0], 'w');
$fh->print('a' x 4096) || print "print failed​: $!\n";
print "errors​: " . ($fh->error() ? "yes ($!)" : "no") . "\n";

When running it on a regular file, it works as expected​:
$ perl foo.pl foo.out
errors​: no

However, when an error actually occurs, the print statement returns an error
but $fh->error does not return true​:
$ perl foo.pl /dev/full
print failed​: No space left on device
errors​: no

IO.xs shows that IO​::Handle​::error just calls PerlIO_error, which typically
just calls PerlIOBase_error to check the PERLIO_F_ERROR flag. However, if I
run gdb on the perl binary, I see that​:

- PerlIO_write is called with (in my case) f=0x81a70e4
- PerlIOBuf_flush correctly sets the flag with the same f=0x81a70e4, and
sets the PERLIO_F_ERROR flag on f->flags
- PerlIO_error gets called with f=0x81a70e4 (from Perl_do_print), and
correctly returns that an error occurred
- PerlIO_error gets called with f=0x81a70e0, whose flags don't have an
error; this is *not* for the error message printed to stdout (I also saw
that happen, and it has a different address).

Whatever the address of the original PerlIO handle is, the final call to
PerlIO_error always has an address 4 bytes less (the size of a pointer on my
platform).

In IO.xs, XS_IO__Handle_error is set up to report errors on the input
stream only, which is why it's seeing the wrong result.
That is to say, it effectively calls PerlIO_error(IoIFP(io)) rather than
IoOFP(io). Anyone have a clue how to handle this, because I dont!

Dave.

--
"Strange women lying in ponds distributing swords is no basis for a system
of government. Supreme executive power derives from a mandate from the
masses, not from some farcical aquatic ceremony."
Dennis - Monty Python and the Holy Grail.

[p5pRT]

From @jkeenan

On Sat Oct 18 04​:58​:34 2003, davem wrote​:

On Mon, Sep 29, 2003 at 03​:43​:41PM -0000, peff-perlbug@​peff.net (via
RT) wrote​:

-----------------------------------------------------------------
The following perl program writes 4096 bytes to a file, checking for
errors​:

$ cat foo.pl
use IO​::File;
my $fh = new IO​::File($ARGV[0], 'w');
$fh->print('a' x 4096) || print "print failed​: $!\n";
print "errors​: " . ($fh->error() ? "yes ($!)" : "no") . "\n";

When running it on a regular file, it works as expected​:
$ perl foo.pl foo.out
errors​: no

However, when an error actually occurs, the print statement returns
an error
but $fh->error does not return true​:
$ perl foo.pl /dev/full
print failed​: No space left on device
errors​: no

Reviewing this older ticket this evening, I could reproduce the OP's
results with Perl 5.12 on Linux/i386​:

#########
$ cat 24067.pl
use IO​::File;
my $fh = new IO​::File($ARGV[0], 'w');
$fh->print('a' x 4096) || print "print failed​: $!\n";
print "errors​: " . ($fh->error() ? "yes ($!)" : "no") . "\n";

$ /usr/local/bin/perl5.12.0 24067.pl /dev/full
print failed​: No space left on device
errors​: no
#########

However, on 5.14 and 5.16, I don't get the print failure message, even
though I still get the "errors​: no"​:

#########
$ /usr/local/bin/perl5.14.0 24067.pl /dev/full
errors​: no

$ /usr/local/bin/perl5.16.0 24067.pl /dev/full
errors​: no
#########

IO.xs shows that IO​::Handle​::error just calls PerlIO_error, which
typically
just calls PerlIOBase_error to check the PERLIO_F_ERROR flag.
However, if I
run gdb on the perl binary, I see that​:

- PerlIO_write is called with (in my case) f=0x81a70e4
- PerlIOBuf_flush correctly sets the flag with the same
f=0x81a70e4, and
sets the PERLIO_F_ERROR flag on f->flags
- PerlIO_error gets called with f=0x81a70e4 (from Perl_do_print),
and
correctly returns that an error occurred
- PerlIO_error gets called with f=0x81a70e0, whose flags don't have
an
error; this is *not* for the error message printed to stdout (I
also saw
that happen, and it has a different address).

Whatever the address of the original PerlIO handle is, the final
call to
PerlIO_error always has an address 4 bytes less (the size of a
pointer on my
platform).

In IO.xs, XS_IO__Handle_error is set up to report errors on the input
stream only, which is why it's seeing the wrong result.
That is to say, it effectively calls PerlIO_error(IoIFP(io)) rather
than
IoOFP(io). Anyone have a clue how to handle this, because I dont!

Dave.

Does anyone know what has changed, for better or worse?

Thank you very much.
Jim Keenan

[p5pRT]

From @arc

James E Keenan via RT <perlbug-followup@​perl.org> wrote​:

On Sat Oct 18 04​:58​:34 2003, davem wrote​:

On Mon, Sep 29, 2003 at 03​:43​:41PM -0000, peff-perlbug@​peff.net (via
RT) wrote​:

The following perl program writes 4096 bytes to a file, checking for errors​:
Reviewing this older ticket this evening, I could reproduce the OP's
results with Perl 5.12 on Linux/i386​:

However, on 5.14 and 5.16, I don't get the print failure message, even
though I still get the "errors​: no"​:

Does anyone know what has changed, for better or worse?

I can reproduce the OP's results under 5.16 by increasing the constant
4096 to 8192. I believe that's because of this commit that first went
into 5.14​:

commit b83080d
Author​: Craig A. Berry <craigberry@​mac.com>
Date​: Thu Feb 17 14​:24​:39 2011 -0600

  Increase PERLIOBUF_DEFAULT_BUFSIZ to larger of 8192 and BUFSIZ.

  The previous default size of a PerlIO buffer (4096 bytes) was
  chosen many years ago before PerlIO was even the default I/O scheme
  for Perl. Benchmarks show that doubling this decade-old default
  increases read and write performance in the neighborhood of 25%
  to 50% when using the default layers of perlio on top of unix.

  The only situation without a noticeable performance benefit so
  far appears to be when physical I/O is so slow that it dwarfs
  any savings from the reduction in layer shuffling, but there
  is also no performance penalty in this case.

  BUFSIZ will be chosen in the unlikely event that it's larger
  than 8192 on the assumption that the system maintainers would
  not set such a value without good reason.

  If the new size causes problems, or to try an even bigger size,
  configure with​:

  ./Configure -Accflags=-DPERLIOBUF_DEFAULT_BUFSIZ=N

  where N is the desired size in bytes; it should probably be a
  multiple of your page size.

That is, when the amount of data written is less than the buffer size,
no OS-level write() call gets issued because of the Perl print. So
the increase in Perl's default buffer size means that a larger print
is needed to trigger this behaviour.

But it does look like a bug to me that $fh->error returns false after
$fh->print has failed.

--
Aaron Crane ** http​://aaroncrane.co.uk/

[p5pRT]

From @peff

On Sun, Sep 30, 2012 at 06​:14​:05AM -0700, Aaron Crane via RT wrote​:

However, on 5.14 and 5.16, I don't get the print failure message, even
though I still get the "errors​: no"​:

Does anyone know what has changed, for better or worse?

I can reproduce the OP's results under 5.16 by increasing the constant
4096 to 8192. I believe that's because of this commit that first went
into 5.14​:

I think my original test was flaky; it should flush explicitly rather
than assuming that print will flush. So this is more realistic for a
program that is trying to be careful about whether output succeeded​:

  use IO​::File;
  my $fh = new IO​::File($ARGV[0], 'w');
  ($fh->print('a' x 4096) && $fh-&gt;flush) || print "print failed​: $!\n";
  print "errors​: " . ($fh-&gt;error() ? "yes ($!)" : "no") . "\n";

But it does look like a bug to me that $fh->error returns false after
$fh->print has failed.

Right, that's the main thing I was trying to report with the test.

-Peff

[p5pRT]

From @jkeenan

On Sat Oct 18 04​:58​:34 2003, davem wrote​:

In IO.xs, XS_IO__Handle_error is set up to report errors on the input
stream only, which is why it's seeing the wrong result.
That is to say, it effectively calls PerlIO_error(IoIFP(io)) rather
than
IoOFP(io). Anyone have a clue how to handle this, because I dont!

This is the source code you're referring to -- correct?

dist/IO/IO.xs
#####

  345 int
  346 ferror(handle)
  347 InputStream handle
  348 CODE​:
  349 if (handle)
  350 #ifdef PerlIO
  351 RETVAL = PerlIO_error(handle);
  352 #else
  353 RETVAL = ferror(handle);
  354 #endif
  355 else {
  356 RETVAL = -1;
  357 errno = EINVAL;
  358 }
  359 OUTPUT​:
  360 RETVAL
#####

[tonycoz]

Perl's I/O objects (that is the IO SV , not PerlIO) can have both an input (PerlIO) file handle and an output file handle. For files that output handle is the same as the input handle, and if you try the error method on a file:

# to a full filesystem
$ ./perl -Ilib -MDevel::Peek -MIO::File -le 'my $fh = new IO::File->new(shift, "w") or die; $fh->print("a" x 4096) && $fh->flush; print $fh->error; close($fh);' /home/tony/mnt/empty.txt
1

For a character device (like /dev/full) or a socket, perl creates a second PerlIO object on the same fd to use as the output handle, and that causes the behaviour this ticket documents.

$ ./perl -Ilib -MDevel::Peek -MIO::File -le 'my $fh = new IO::File->new(shift, "w") or die; $fh->print("a" x 4096) && $fh->flush; print $fh->error; close($fh);' /dev/full
0

The separate handle allows sockets and character devices to have separate input and output buffers, but it does make this method less useful.

[peff]

@tonycoz Yeah, that makes sense. Since the fact that one IO object is represented under the hood as two individual PerlIO handles is an implementation detail not known to callers of IO, would it be reasonable for $fh->error to return PerlIO_error(input_handle) || PerlIO_error(output_handle)? I think that would match the single-handle case more closely (where an error on either side would cause the handle to report an error).

It's possible that some existing callers using sockets could be relying on ignoring output errors, but it seems unlikely (and I think it could be argued they're in the wrong).

[tonycoz]

I'm working on such a PR now, I also want to check clearerr() works here.