BBC: RJBS/Number-Format-1.76.tar.gz fails under German locale

[andk]

Description

With v5.37.3-160-g818cdb7aa9 tests started to fail for RJBS/Number-Format-1.76.tar.gz under the locale de_DE.utf8.

Sample fail reports:

• with 5.37.4: http://www.cpantesters.org/cpan/report/035a4956-13eb-11ee-87a2-9a761ae52feb
• with 5.38.0 RC2: http://www.cpantesters.org/cpan/report/50c40856-13e8-11ee-b12d-5b681ae52feb

The commit has already been discussed in #20617

Steps to Reproduce

LC_NUMERIC=de_DE.utf8 cpan -i RJBS/Number-Format-1.76.tar.gz

Expected behavior

Should build, test and install the pkg

Perl configuration

# perl -V output goes here
Summary of my perl5 (revision 5 version 37 subversion 4) configuration:
  Commit id: f2582f5b18658f945a763f2edc110cdc7c5220e7
  Platform:
    osname=linux
    osvers=5.19.0-1-amd64
    archname=x86_64-linux
    uname='linux k93msid 5.19.0-1-amd64 #1 smp preempt_dynamic debian 5.19.6-1 (2022-09-01) x86_64 gnulinux '
    config_args='-Dprefix=/home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.37.4/da1c -Dmyhostname=k93msid -Dinstallusrbinperl=n -Uversiononly -Dusedevel -des -Ui_db -Dlibswanted=cl pthread socket inet nsl gdbm dbm malloc dl ld sun m crypt sec util c cposix posix ucb BSD gdbm_compat -Uuseithreads -Uuselongdouble -DEBUGGING=-g'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O2 -g'
    cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='12.2.0'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/x86_64-linux-gnu /usr/lib /usr/lib64
    libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=/lib/x86_64-linux-gnu/libc.so.6
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.34'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_SIPHASH13
    PERL_HASH_USE_SBOX32
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_DEVEL
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
  Built under linux
  Compiled at Sep 22 2022 09:26:30
  %ENV:
    PERL5LIB=""
    PERL5OPT=""
    PERL5_CPANPLUS_IS_RUNNING="2831720"
    PERL5_CPAN_IS_RUNNING="2831720"
    PERL_CANARY_STABILITY_NOPROMPT="1"
    PERL_MM_USE_DEFAULT="1"
    PERL_USE_UNSAFE_INC="1"
  @INC:
    /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.37.4/da1c/lib/site_perl/5.37.4/x86_64-linux
    /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.37.4/da1c/lib/site_perl/5.37.4
    /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.37.4/da1c/lib/5.37.4/x86_64-linux
    /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.37.4/da1c/lib/5.37.4
    .

[jkeenan]

Description

With v5.37.3-160-g818cdb7aa9 tests started to fail for RJBS/Number-Format-1.76.tar.gz under the locale de_DE.utf8.

Sample fail reports:

* with 5.37.4: http://www.cpantesters.org/cpan/report/035a4956-13eb-11ee-87a2-9a761ae52feb

* with 5.38.0 RC2: http://www.cpantesters.org/cpan/report/50c40856-13e8-11ee-b12d-5b681ae52feb

The commit has already been discussed in #20617

Steps to Reproduce

LC_NUMERIC=de_DE.utf8 cpan -i RJBS/Number-Format-1.76.tar.gz

@andk I'm a bit puzzled by this. You report test failures with de_DE.utf8. However, we're also getting different test failures in the same file, t/locale.t, from a different tester who, AFAICT, is not setting any envvars like LC_NUMERIC. See, e.g., http://www.cpantesters.org/cpan/report/581eb232-0f78-11ee-852f-8e72b69fa997. (We're also getting PASSes on this distro from other CPANtesters, including myself.)

From that I would infer that this distro has problems, as yet undiagnosed, above and beyond de_DE.utf8. Is that a valid inference?

[andk]

I don't think that it is so unusual, especially for locale issues: when a test uncovers
a problem with one locale, it certainly is possible that the same test in a different
locale passes; and it is also possible that the test fails on yet another
locale. The problem is to find out how many bugs are relevant to explain the issue and
where they are located.

As a help for further containment I should add that I have sent FAIL reports on Debian
and Ubuntu, both in German locale, in both cases only the LC_NUMERIC category was
relevant. In the link you provide, it seems that the operating system was Alpine and the
locale was C.UTF-8 and LC_COLLATE was set to C. I have no Alpine system available, so
cannot contribute to this area.

[khwilliamson]

On 6/27/23 07:43, andk wrote: I don't think that it is so unusual, especially for locale issues: when a test uncovers a problem with one locale, it certainly is possible that the same test in a different locale passes; and it is also possible that the test fails on yet another locale. The problem is to find out how many bugs are relevant to explain the issue and where they are located. As a help for further containment I should add that I have sent FAIL reports on Debian and Ubuntu, both in German locale, in both cases only the LC_NUMERIC category was relevant.

I am unable to reproduce this on my Ubuntu system. This bug likely would occur on any locale whose decimal radix character is a comma. Also, I suspect the issue comes from Andreas having a default environment different than mine. Please run env | egrep '^L[AC]' and report back the results In the link you provide, it seems that the operating system

was Alpine and the locale was C.UTF-8 and LC_COLLATE was set to C. I have no Alpine system available, so cannot contribute to this area.

Alpine ships with crippled locales. You need to install an extra package to enable them. I have a patch for 5.39 that avoids test failures with the crippled version.

…

— Reply to this email directly, view it on GitHub <#21178 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA2DH6XTMN474XZZDQ7USLXNLPP7ANCNFSM6AAAAAAZUMIPGY>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[andk]

As requested:

>sand@k93jammy:/tmp/loop_over_bdir-bQg7EB/Number-Format-1.76-0% env | egrep '^L[AC]'
LC_NUMERIC=de_DE.utf8
>sand@k93jammy:/tmp/loop_over_bdir-bQg7EB/Number-Format-1.76-0% make test TEST_FILES=t/locale.t
PERL_DL_NONLAZY=1 "/tmp/basesmoker-reloperl-eB6f/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/locale.t
t/locale.t .. 1/? 
#   Failed test 'unformat German'
#   at t/locale.t line 38.
#          got: '123456'
#     expected: '123456,79'
# Looks like you failed 1 test of 9.
t/locale.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/9 subtests 
        (less 1 skipped subtest: 7 okay)

Test Summary Report
-------------------
t/locale.t (Wstat: 256 (exited 1) Tests: 9 Failed: 1)
  Failed test:  4
  Non-zero exit status: 1
Files=1, Tests=9,  0 wallclock secs ( 0.03 usr  0.00 sys +  0.05 cusr  0.01 csys =  0.09 CPU)
Result: FAIL
Failed 1/1 test programs. 1/9 subtests failed.
make: *** [Makefile:862: test_dynamic] Error 1
zsh: exit 2     make test TEST_FILES=t/locale.t

[jkeenan]

On 6/27/23 07:43, andk wrote: I don't think that it is so unusual, especially for locale issues: when a test uncovers a problem with one locale, it certainly is possible that the same test in a different locale passes; and it is also possible that the test fails on yet another locale. The problem is to find out how many bugs are relevant to explain the issue and where they are located. As a help for further containment I should add that I have sent FAIL reports on Debian and Ubuntu, both in German locale, in both cases only the LC_NUMERIC category was relevant.
I am unable to reproduce this on my Ubuntu system. This bug likely would occur on any locale whose decimal radix character is a comma. Also, I suspect the issue comes from Andreas having a default environment different than mine. Please run env | egrep '^L[AC]' and report back the results In the link you provide, it seems that the operating system
was Alpine and the locale was C.UTF-8 and LC_COLLATE was set to C. I have no Alpine system available, so cannot contribute to this area.
Alpine ships with crippled locales. You need to install an extra package to enable them. I have a patch for 5.39 that avoids test failures with the crippled version.
…
— Reply to this email directly, view it on GitHub <#21178 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA2DH6XTMN474XZZDQ7USLXNLPP7ANCNFSM6AAAAAAZUMIPGY. You are receiving this because you are subscribed to this thread.Message ID: @.***>

@khwilliamson, @andk responded to your request for information about his default locales. Also, you mentioned something about having a patch in progress to handle Alpine Linux properly. Can you give us an update on the issues raised in this ticket?

[jkeenan]

On 6/27/23 07:43, andk wrote: I don't think that it is so unusual, especially for locale issues: when a test uncovers a problem with one locale, it certainly is possible that the same test in a different locale passes; and it is also possible that the test fails on yet another locale. The problem is to find out how many bugs are relevant to explain the issue and where they are located. As a help for further containment I should add that I have sent FAIL reports on Debian and Ubuntu, both in German locale, in both cases only the LC_NUMERIC category was relevant.
I am unable to reproduce this on my Ubuntu system. This bug likely would occur on any locale whose decimal radix character is a comma. Also, I suspect the issue comes from Andreas having a default environment different than mine. Please run env | egrep '^L[AC]' and report back the results In the link you provide, it seems that the operating system
was Alpine and the locale was C.UTF-8 and LC_COLLATE was set to C. I have no Alpine system available, so cannot contribute to this area.
Alpine ships with crippled locales. You need to install an extra package to enable them. I have a patch for 5.39 that avoids test failures with the crippled version.
…
— Reply to this email directly, view it on GitHub <#21178 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA2DH6XTMN474XZZDQ7USLXNLPP7ANCNFSM6AAAAAAZUMIPGY. You are receiving this because you are subscribed to this thread.Message ID: @.***>

@khwilliamson, @andk responded to your request for information about his default locales. Also, you mentioned something about having a patch in progress to handle Alpine Linux properly. Can you give us an update on the issues raised in this ticket?

@khwilliamson can you comment on the status of this ticket? Thanks.

[khwilliamson]

I think it may be fixed. Hopefully, @andk could try now on blead or the latest 5.39, and report back.

[khwilliamson]

@andk What did you find out?