Assertion failure in Perl_av_push/S_handle_regex_sets

[p5pRT]

Migrated from rt.perl.org#133889 (status was 'resolved')

Searchable as RT133889$

[p5pRT]

From @dur-randir

Created by @dur-randir

While fuzzing perl v5.29.8-21-gde59f38ed9 built with afl and run
under libdislocator, I found the following program

s>(?[\P{Is0}>>

to cause an assertion failure​:

perl​: av.c​:611​: Perl_av_push​: Assertion `val' failed.

GDB stack trace is following​:

#0 __GI_raise (sig=sig@​entry=6) at ../sysdeps/unix/sysv/linux/raise.c​:50
#1 0x00007ffff7c25535 in __GI_abort () at abort.c​:79
#2 0x00007ffff7c2540f in __assert_fail_base (fmt=0x7ffff7d87ee0
"%s%s%s​:%u​: %s%sAssertion `%s' failed.\n%n", assertion=0x555555a85d7b
"val",
  file=0x555555a85b16 "av.c", line=611, function=<optimized out>) at
assert.c​:92
#3 0x00007ffff7c330f2 in __GI___assert_fail (assertion=0x555555a85d7b
"val", file=0x555555a85b16 "av.c", line=611,
  function=0x555555a85e98 <__PRETTY_FUNCTION__.18075>
"Perl_av_push") at assert.c​:101
#4 0x00005555557421cd in Perl_av_push (av=0x555555b4b110, val=0x0) at av.c​:611
#5 0x00005555556c1214 in S_handle_regex_sets
(pRExC_state=0x7fffffffd690, return_invlist=0x0, flagp=0x7fffffffcaf8,
depth=6,
  oregcomp_parse=0x555555b4f611 "?[\\P{Is0}") at regcomp.c​:16065
#6 0x00005555556a5078 in S_reg (pRExC_state=0x7fffffffd690, paren=91,
flagp=0x7fffffffcaf8, depth=5) at regcomp.c​:11917
#7 0x00005555556af40f in S_regatom (pRExC_state=0x7fffffffd690,
flagp=0x7fffffffcd84, depth=4) at regcomp.c​:13155
#8 0x00005555556a8d11 in S_regpiece (pRExC_state=0x7fffffffd690,
flagp=0x7fffffffcea0, depth=3) at regcomp.c​:12372
#9 0x00005555556a85f9 in S_regbranch (pRExC_state=0x7fffffffd690,
flagp=0x7fffffffcf44, first=1, depth=2) at regcomp.c​:12290
#10 0x00005555556a5f5b in S_reg (pRExC_state=0x7fffffffd690, paren=0,
flagp=0x7fffffffd3c8, depth=1) at regcomp.c​:12011
#11 0x000055555568a1b4 in Perl_re_op_compile (patternp=0x0,
pat_count=1, expr=0x555555b4f508, eng=0x555555b19d20
<PL_core_reg_engine>, old_re=0x0,
  is_bare_re=0x0, orig_rx_flags=0, pm_flags=0) at regcomp.c​:7665
#12 0x00005555555a7080 in Perl_pmruntime (o=0x555555b4f548,
expr=0x555555b4f508, repl=0x555555b4f4c8, flags=1, floor=0) at
op.c​:7094
#13 0x000055555565d51c in Perl_yyparse (gramtype=258) at perly.y​:1228
#14 0x00005555555d92ea in S_parse_body (env=0x0, xsinit=0x55555558e1d8
<xs_init>) at perl.c​:2507
#15 0x00005555555d75bc in perl_parse (my_perl=0x555555b24260,
xsinit=0x55555558e1d8 <xs_init>, argc=3, argv=0x7fffffffe1d8, env=0x0)
at perl.c​:1798
#16 0x000055555558e11b in main (argc=3, argv=0x7fffffffe1d8,
env=0x7fffffffe1f8) at perlmain.c​:126

This is a regression in blead, bisect points to

commit 73b9584
Author​: Karl Williamson <khw@​cpan.org>
Date​: Mon Aug 20 18​:31​:04 2018 -0600

  Move \p{user-defined} to core from utf8_heavy.pl

  This large commit moves the handling of user-defined properties to C
  code. This should speed it up, but the main reason to do this is to
  stop using swashes in this case, leaving only tr/// using them. Once
  that too is converted, all swash handling can be ripped out of perl.

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.29.9:

Configured by dur-randir at Wed Feb 27 14:51:01 MSK 2019.

Summary of my perl5 (revision 5 version 29 subversion 9) configuration:
  Commit id: c1e47bad34ce1d9c84ed57c9b8978bcbd5a02e98
  Platform:
    osname=darwin
    osvers=13.4.0
    archname=darwin-thread-multi-2level
    uname='darwin isengard.local 13.4.0 darwin kernel version 13.4.0:
mon jan 11 18:17:34 pst 2016; root:xnu-2422.115.15~1release_x86_64
x86_64 '
    config_args='-de -Dusedevel -DDEBUGGING -Dusethreads'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include -DPERL_USE_SAFE_PUTENV'
    optimize='-O3 -g'
    cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -mmacosx-version-min=10.9 -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
/usr/lib
    libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=
    so=dylib
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -mmacosx-version-min=10.9 -bundle -undefined
dynamic_lookup -L/usr/local/lib -fstack-protector'



@INC for perl 5.29.9:
    lib
    /usr/local/lib/perl5/site_perl/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/site_perl/5.29.9
    /usr/local/lib/perl5/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/5.29.9


Environment for perl 5.29.9:
    DYLD_LIBRARY_PATH (unset)
    HOME=/Users/dur-randir
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin
    PERLBREW_HOME=/Users/dur-randir/.perlbrew
    PERLBREW_MANPATH=/Users/dur-randir/perlbrew/perls/perl-5.22.1/man
    PERLBREW_PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin
    PERLBREW_PERL=perl-5.22.1
    PERLBREW_ROOT=/Users/dur-randir/perlbrew
    PERLBREW_SHELLRC_VERSION=0.84
    PERLBREW_VERSION=0.84
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/zsh

[p5pRT]

@khwilliamson - Status changed from 'new' to 'open'

[p5pRT]

From @khwilliamson

Fixed by
commit 62286d5
Author​: Karl Williamson <khw@​cpan.org>
Date​: Fri Mar 22 11​:32​:32 2019 -0600

  PATCH​: [perl #133889] Assertion failure
 
  I did not bisect this, but this is a regression.
 
  This code is using a user-defined property that isn't defined. It
  should catch that and, since this is within regex sets, quit, but
  instead continues and ends up using an undefined value.
--
Karl Williamson

[p5pRT]

@khwilliamson - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'