stat.t build tests failing on some Cygwin

[p5pRT]

Migrated from rt.perl.org#127953 (status was 'open')

Searchable as RT127953$

[p5pRT]

From @jmaslak

Created by @jmaslak

This is a bug report for perl from jmaslak@​antelope.net,
generated with the help of perlbug 1.40 running under perl 5.22.1.

-----------------------------------------------------------------
On at least Perl 5.22.1 and Perl 5.24.0-RC0, Cygwin builds under
perlbrew as a standard Windows 10 user fails. User is a member
of the Administrators group on the local machine, but Cygwin
and Perl.exe are not running with any special privileges.

t/op/stat.t fails on this environment​:

ok 88 - unlink tmp file
ok 89 - can create temp file
ok 90 - -T _ doesn't break the statbuffer
not ok 91 - -T _ doesn't break lstat for unreadable file
# Failed test 91 - -T _ doesn't break lstat for unreadable file at
# t/op/stat.t line 552

The problem is that Perl is essentually doing (line numbers are from
5.22.1)​:

534 ok(open(F, ">", $tmpfile), 'can create temp file');
535 close F;
536 chmod 0077, $tmpfile;
537 my @​a = stat($tmpfile);
538 my $s1 = -s _;
539 -T _;
540 my $s2 = -s _;
541 is($s1, $s2, q(-T _ doesn't break the statbuffer));
542 SKIP​: {
543 ⇥ my $root_uid = $Is_Cygwin ? 18 : 0;
544 ⇥ skip "No lstat", 1 unless $Config{d_lstat};
545 ⇥ skip "uid=0", 1 if $< == $root_uid or $&gt; == $root_uid;
546 ⇥ skip "Can't check if admin user in miniperl", 1
547 ⇥ if $^O =~ /^(cygwin|MSWin32|msys)$/ && is_miniperl();
548 ⇥ skip "Readable by group/other means readable by me on $^O", 1 if $^O eq 'VMS'
549 or ($^O =~ /^(cygwin|MSWin32|msys)$/ and Win32​::IsAdminUser());
550 ⇥ lstat($tmpfile);
551 ⇥ -T _;
552 ⇥ ok(eval { lstat _ },
553 ⇥ q(-T _ doesn't break lstat for unreadable file));
554 }

Line 536 on Cygwin Windows 10 doesn't operate as would be expected. Here's
an example from the command line (tests were done from inside the build
directory, on an NTFS partition)​:

# I'm an admin, but Win32​::IsAdminUser() doesn't detect that because I'm
# not running in an elevated privilege context. So I don't have admin
# rights in this session

$ id
uid=181623(jmasl) gid=181623(jmasl) groups=197609(jmasl),401408(Medium Mandatory Level),545(Users),4(INTERACTIVE),66049(CONSOLE LOGON),11(Authenticated Users),15(This Organization),68413(MicrosoftAccount+jmaslak@​antelope.net),113(Local account),66048(LOCAL),262180(Cloud Account Authentication)
$ PERL5LIB=lib ./perl.exe -MWin32 -e 'print Win32​::IsAdminUser(), "\n"'
0

# When I create a file and set it's mode to 077, the mode instead ends
# up being 777, which I can read​:

$ echo test >test.txt
$ ls -l test.txt
-rw-r--r-- 1 jmasl jmasl 5 Apr 21 21​:04 test.txt
$ chmod 077 test.txt
$ ls -l test.txt
-rwxrwxrwx 1 jmasl jmasl 5 Apr 21 21​:04 test.txt
$ cat test.txt
test

# If I set the mode to 000, I can't.

$ chmod 000 test.txt
joel-win10​:perl-5.22.1$ ls -l test.txt
---------- 1 jmasl jmasl 5 Apr 21 21​:04 test.txt
joel-win10​:perl-5.22.1$ cat test.txt
cat​: test.txt​: Permission denied

As you can see, chmod 077, at least on my Win 10 machine does let you
read the file still, thus the -T can successfully read the file. That
means the lstat _ in the eval at 552 succeeds. If the file was not
readable, it would error with "The stat preceding lstat() wasn't an lstat"
and thus the test passes.

I think the best way to fix this in the test is probably to skip this
test period if the user is running in Cygwin. Alternatively, use a
mode of 0000 instead of 0077 in the chmod() statement. The build fails
as a normal user today, but passes as an admin because we skip the test
if an admin (with elevated privileges) does the build.

The Perl I used for perlbug is not the Perl that is causing issues -
the issue is on Cygwin on newer Windows with Perls up to 5.24.0-RC1.

Perl Info

Flags:
    category=install
    severity=medium

Site configuration information for perl 5.22.1:

Configured by jmaslak at Wed Dec 30 17:06:04 MST 2015.

Summary of my perl5 (revision 5 version 22 subversion 1) configuration:
   
  Platform:
    osname=linux, osvers=3.19.0-30-generic, archname=x86_64-linux
    uname='linux red 3.19.0-30-generic #34-ubuntu smp fri oct 2 22:08:41 utc 2015 x86_64 x86_64 x86_64 gnulinux '
    config_args='-de -Dprefix=/home/jmaslak/perl5/perlbrew/perls/perl-5.22.1 -Aeval:scriptdir=/home/jmaslak/perl5/perlbrew/perls/perl-5.22.1/bin'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion='', gccversion='4.9.2', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.21.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.21'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

Locally applied patches:
    Devel::PatchPerl 1.38


@INC for perl 5.22.1:
    /home/jmaslak/perl5/perlbrew/perls/perl-5.22.1/lib/site_perl/5.22.1/x86_64-linux
    /home/jmaslak/perl5/perlbrew/perls/perl-5.22.1/lib/site_perl/5.22.1
    /home/jmaslak/perl5/perlbrew/perls/perl-5.22.1/lib/5.22.1/x86_64-linux
    /home/jmaslak/perl5/perlbrew/perls/perl-5.22.1/lib/5.22.1
    .


Environment for perl 5.22.1:
    HOME=/home/jmaslak
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH=:/usr/local/cuda-7.0/lib64
    LOGDIR (unset)
    PATH=/home/jmaslak/bin:~/bin:/home/jmaslak/perl5/perlbrew/bin:/home/jmaslak/perl5/perlbrew/perls/perl-5.22.1/bin:/home/jmaslak/.rakudobrew/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/cuda-7.0/bin:/usr/sbin
    PERLBREW_BASHRC_VERSION=0.74
    PERLBREW_HOME=/home/jmaslak/.perlbrew
    PERLBREW_MANPATH=/home/jmaslak/perl5/perlbrew/perls/perl-5.22.1/man
    PERLBREW_PATH=/home/jmaslak/perl5/perlbrew/bin:/home/jmaslak/perl5/perlbrew/perls/perl-5.22.1/bin
    PERLBREW_PERL=perl-5.22.1
    PERLBREW_ROOT=/home/jmaslak/perl5/perlbrew
    PERLBREW_VERSION=0.74
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From @tonycoz

On Thu Apr 21 20​:25​:29 2016, jmaslak@​antelope.net wrote​:

On at least Perl 5.22.1 and Perl 5.24.0-RC0, Cygwin builds under
perlbrew as a standard Windows 10 user fails. User is a member
of the Administrators group on the local machine, but Cygwin
and Perl.exe are not running with any special privileges.

t/op/stat.t fails on this environment​:

ok 88 - unlink tmp file
ok 89 - can create temp file
ok 90 - -T _ doesn't break the statbuffer
not ok 91 - -T _ doesn't break lstat for unreadable file
# Failed test 91 - -T _ doesn't break lstat for unreadable file at
# t/op/stat.t line 552

Works fine here​:

tony@​mimas ~/dev/perl/git/perl
$ cd t

tony@​mimas ~/dev/perl/git/perl/t
$ ./perl harness op/stat.t
op/stat.t .. ok
All tests successful.
Files=1, Tests=118, 6 wallclock secs ( 0.01 usr 0.00 sys + 0.09 cusr 0.34 csys = 0.45 CPU)
Result​: PASS

tony@​mimas ~/dev/perl/git/perl/t
$ uname -a
CYGWIN_NT-10.0-WOW mimas 2.5.0(0.297/5/3) 2016-03-29 15​:44 i686 Cygwin

tony@​mimas ~/dev/perl/git/perl/t
$ id
uid=1001(tony) gid=513(None) groups=513(None),545(Users),14(REMOTE INTERACTIVE LOGON),4(INTERACTIVE),11(Authenticated Users),15(This Organization),113(Local account),66048(LOCAL),262154(NTLM Authentication),401408(Medium Mandatory Level)

tony@​mimas ~/dev/perl/git/perl/t
$ ./perl -v

This is perl 5, version 24, subversion 0 (v5.24.0-RC2) built for cygwin-thread-multi-64int
(with 1 registered patch, see perl -V for more detail)

The user is an admin but not in an admin shell.

# I'm an admin, but Win32​::IsAdminUser() doesn't detect that because
I'm
# not running in an elevated privilege context. So I don't have admin
# rights in this session

$ id
uid=181623(jmasl) gid=181623(jmasl) groups=197609(jmasl),401408(Medium
Mandatory Level),545(Users),4(INTERACTIVE),66049(CONSOLE
LOGON),11(Authenticated Users),15(This
Organization),68413(MicrosoftAccount+jmaslak@​antelope.net),113(Local
account),66048(LOCAL),262180(Cloud Account Authentication)

I wonder if it's the cloud authentication.

$ PERL5LIB=lib ./perl.exe -MWin32 -e 'print Win32​::IsAdminUser(),
"\n"'
0

# When I create a file and set it's mode to 077, the mode instead ends
# up being 777, which I can read​:

$ echo test >test.txt
$ ls -l test.txt
-rw-r--r-- 1 jmasl jmasl 5 Apr 21 21​:04 test.txt
$ chmod 077 test.txt
$ ls -l test.txt
-rwxrwxrwx 1 jmasl jmasl 5 Apr 21 21​:04 test.txt
$ cat test.txt
test

tony@​mimas ~/dev/perl/git/perl
$ echo test >test.txt

tony@​mimas ~/dev/perl/git/perl
$ ls -l test.txt
-rw-r--r-- 1 tony None 5 Apr 25 10​:46 test.txt

tony@​mimas ~/dev/perl/git/perl
$ chmod 077 test.txt

tony@​mimas ~/dev/perl/git/perl
$ ls -l test.txt
----rwxrwx 1 tony None 5 Apr 25 10​:46 test.txt

tony@​mimas ~/dev/perl/git/perl
$ chmod 000 test.txt

tony@​mimas ~/dev/perl/git/perl
$ ls -l test.txt
---------- 1 tony None 5 Apr 25 10​:46 test.txt

# If I set the mode to 000, I can't.

$ chmod 000 test.txt
joel-win10​:perl-5.22.1$ ls -l test.txt
---------- 1 jmasl jmasl 5 Apr 21 21​:04 test.txt
joel-win10​:perl-5.22.1$ cat test.txt
cat​: test.txt​: Permission denied

As you can see, chmod 077, at least on my Win 10 machine does let you
read the file still, thus the -T can successfully read the file. That
means the lstat _ in the eval at 552 succeeds. If the file was not
readable, it would error with "The stat preceding lstat() wasn't an
lstat"
and thus the test passes.

I think the best way to fix this in the test is probably to skip this
test period if the user is running in Cygwin. Alternatively, use a
mode of 0000 instead of 0077 in the chmod() statement. The build
fails
as a normal user today, but passes as an admin because we skip the
test
if an admin (with elevated privileges) does the build.

I test cygwin regularly as a non-admin user (a user who is not a member of administrators.)

The Perl I used for perlbug is not the Perl that is causing issues -
the issue is on Cygwin on newer Windows with Perls up to 5.24.0-RC1.

perl -V from your cygwin build would have been usefui to see what release of Cygwin you're running.

Tony

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @jmaslak

Here's my perl -V for a build of perl-5.24.0-RC1 (built without executing
tests). I'll log in as a standard user (non-cloud) in a bit and try some
tests there as well.

Summary of my perl5 (revision 5 version 24 subversion 0) configuration​:

  Platform​:
  osname=cygwin, osvers=2.5.0(0.29753), archname=cygwin-thread-multi
  uname='cygwin_nt-10.0 joel-win10 2.5.0(0.29753) 2016-04-11 09​:58 x86_64
cygwin '
  config_args='-de
-Dprefix=/home/jmasl/perl5/perlbrew/perls/perl-5.24.0-RC1
-Aeval​:scriptdir=/home/jmasl/perl5/perlbrew/perls/perl-5.24.0-RC1/bin'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='gcc', ccflags ='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -fwrapv
-fno-strict-aliasing -pipe -fstack-protector-strong -D_FORTIFY_SOURCE=2',
  optimize='-O3',
  cppflags='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -fwrapv
-fno-strict-aliasing -pipe -fstack-protector-strong'
  ccversion='', gccversion='5.3.0', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678,
doublekind=3
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16,
longdblkind=3
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='g++', ldflags =' -Wl,--enable-auto-import -Wl,--export-all-symbols
-Wl,--enable-auto-image-base -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/lib /usr/lib/../lib/../include/w32api /usr/local/lib /lib
  libs=-lpthread -ldl -lcrypt
  perllibs=-lpthread -ldl -lcrypt
  libc=/usr/lib/libc.a, so=dll, useshrplib=true, libperl=cygperl5_24_0.dll
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
  cccdlflags=' ', lddlflags=' --shared -Wl,--enable-auto-import
-Wl,--export-all-symbols -Wl,--enable-auto-image-base -L/usr/local/lib
-fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
  PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV
  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
  PERL_IMPLICIT_CONTEXT PERL_PRESERVE_IVUV
  PERL_USE_SAFE_PUTENV USE_64_BIT_ALL USE_64_BIT_INT
  USE_ITHREADS USE_LARGE_FILES USE_LOCALE
  USE_LOCALE_COLLATE USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
  USE_PERL_ATOF USE_REENTRANT_API
  Locally applied patches​:
  RC1
  Devel​::PatchPerl 1.38
  Built under cygwin
  Compiled at Apr 21 2016 21​:59​:54
  %ENV​:
  PERLBREW_BASHRC_VERSION="0.75"
  PERLBREW_HOME="/home/jmasl/.perlbrew"
  PERLBREW_MANPATH="/home/jmasl/perl5/perlbrew/perls/perl-5.24.0-RC1/man"

PERLBREW_PATH="/home/jmasl/perl5/perlbrew/bin​:/home/jmasl/perl5/perlbrew/perls/perl-5.24.0-RC1/bin"
  PERLBREW_PERL="perl-5.24.0-RC1"
  PERLBREW_ROOT="/home/jmasl/perl5/perlbrew"
  PERLBREW_VERSION="0.75"
  @​INC​:

/home/jmasl/perl5/perlbrew/perls/perl-5.24.0-RC1/lib/site_perl/5.24.0/cygwin-thread-multi
  /home/jmasl/perl5/perlbrew/perls/perl-5.24.0-RC1/lib/site_perl/5.24.0

/home/jmasl/perl5/perlbrew/perls/perl-5.24.0-RC1/lib/5.24.0/cygwin-thread-multi
  /home/jmasl/perl5/perlbrew/perls/perl-5.24.0-RC1/lib/5.24.0
  .

[p5pRT]

From @jmaslak

Sure enough, I created a local user ("testuser") without admin rights and
was able to compile 5.24.0-RC2 just fine on Cygwin. I also validated that
chmod 077 did what was expected. Your observation was awesome and let me
find the issue - thanks for the help!

It looks like the issue was that Cygwin was using a fake "jmasl" group for
the cloud auth account's primary POSIX group, but was using the "None"
group for the local user's primary group. I added an /etc/passwd entry for
jmasl and set jmasl's primary group to the user's group (which I populated
/etc/group with).

So I have a workaround for this now that works for me. I'll put in a bug
report with Cygwin when I have a bit of time to document this issue. It
might not be a bad idea to document this in README.cygwin - I'll probably
try to submit a patch to the docs in the next few weeks.

[p5pRT]

From Stromeko@nexgo.de

Joel Maslak writes​:

It looks like the issue was that Cygwin was using a fake "jmasl" group for
the cloud auth account's primary POSIX group, but was using the "None"
group for the local user's primary group. I added an /etc/passwd entry for
jmasl and set jmasl's primary group to the user's group (which I populated
/etc/group with).

No, that's what you get from Windows when you use one of their
"Microsoft Accounts". They actually use the same SID for both the
account and the primary group in your security token (something that's
not possible to do for any other account).

So I have a workaround for this now that works for me. I'll put in a bug
report with Cygwin when I have a bit of time to document this issue. It
might not be a bad idea to document this in README.cygwin - I'll probably
try to submit a patch to the docs in the next few weeks.

Perhaps you try the latest Cygwin version first.

Regards,
Achim.
--
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9​:
http​://Synth.Stromeko.net/Downloads.html#KorgSDada