S_make_trie(RExC_state_t *, regnode *, regnode *, regnode *, regnode *, U32, U32, U32): Assertion `*uc != LATIN_SMALL_LETTER_SHARP_S' failed

[dur-randir]

This is a bug report for perl from sergey.aleynikov@gmail.com,
generated with the help of perlbug 1.41 running under perl 5.31.6.

[Please describe your issue here]

While fuzzing perl v5.31.5-213-g9bec17d7c built with afl and run
under libdislocator, I found the following program

0=~/\p{nv=\\\}(?0)|\337ss|\337ss/

to cause an assertion failure on debugging builds

perl: regcomp.c:2820: I32 S_make_trie(RExC_state_t *, regnode *, regnode *, regnode *, regnode *, U32, U32, U32): Assertion `*uc != LATIN_SMALL_LETTER_SHARP_S' failed.

GDB stack strace is

#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x00007ffff7c24535 in __GI_abort () at abort.c:79
#2 0x00007ffff7c2440f in __assert_fail_base (fmt=0x7ffff7d86ee0 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=0xf9b900 <.str.481> "*uc != LATIN_SMALL_LETTER_SHARP_S", file=0xf469a0 <.str.4> "regcomp.c", line=2820, function=)
at assert.c:92
#3 0x00007ffff7c32102 in __GI___assert_fail (assertion=0xf9b900 <.str.481> "*uc != LATIN_SMALL_LETTER_SHARP_S", file=0xf469a0 <.str.4> "regcomp.c",
line=2820,
function=0xf9b6c0 <PRETTY_FUNCTION.S_make_trie> "I32 S_make_trie(RExC_state_t *, regnode *, regnode *, regnode *, regnode *, U32, U32, U32)")
at assert.c:101
#4 0x000000000083b986 in S_make_trie (pRExC_state=0x7fffffffa790, startbranch=0x60b000000a0c, first=0x60b000000a24, last=0x60b000000a3c,
tail=0x60b000000a40, word_count=2, flags=, depth=) at regcomp.c:2820
#5 0x00000000007cfbe0 in S_study_chunk (pRExC_state=, scanp=, minlenp=, deltap=,
last=0x60b000000a44, data=, stopparen=-1, recursed_depth=, and_withp=, flags=9216, depth=0)
at regcomp.c:5024
#6 0x0000000000787c68 in Perl_re_op_compile (patternp=, pat_count=, expr=, eng=, old_re=0x0,
is_bare_re=, orig_rx_flags=, pm_flags=) at regcomp.c:8092
#7 0x000000000081a304 in Perl_re_compile (pattern=, rx_flags=0) at regcomp.c:6575
#8 Perl_parse_uniprop_string (name=0x603000000bb3 "nv=\\\}(?0)|\337ss|\337ss", name_len=6, is_utf8=false, to_fold=false, runtime=false,
deferrable=true, user_defined_ptr=, msg=, level=) at regcomp.c:22934
#9 0x000000000088ceb2 in S_regclass (pRExC_state=0x7fffffffcc50, flagp=0x7fffffffc500, depth=5, stop_at_1=true, allow_mutiple_chars=false,
silence_non_portable=false, strict=, optimizable=, ret_invlist=) at regcomp.c:17210
#10 0x0000000000867449 in S_regatom (pRExC_state=0x7fffffffcc50, flagp=0x7fffffffc500, depth=4) at regcomp.c:13538
#11 0x0000000000849c03 in S_regpiece (pRExC_state=0x7fffffffcc50, depth=3, flagp=) at regcomp.c:12404
#12 S_regbranch (pRExC_state=0x7fffffffcc50, flagp=0x7fffffffc800, first=, depth=0) at regcomp.c:12324
#13 0x00000000007a60df in S_reg (pRExC_state=0x7fffffffcc50, paren=58, flagp=0x7fffffffcb20, depth=1) at regcomp.c:12026
#14 0x0000000000781290 in Perl_re_op_compile (patternp=, pat_count=, expr=, eng=, old_re=0x0,
is_bare_re=, orig_rx_flags=, pm_flags=) at regcomp.c:7738
#15 0x000000000055c407 in Perl_pmruntime (o=0x615000000468, expr=0x615000000430, repl=0x0, flags=1, floor=) at op.c:8101
#16 0x0000000000752b06 in Perl_yyparse (gramtype=) at perly.y:1260
#17 0x0000000000614c8d in S_parse_body (env=, xsinit=) at perl.c:2529
#18 0x000000000060aa47 in perl_parse (my_perl=, xsinit=, argc=, argv=, env=0x0) at perl.c:1820
#19 0x00000000005352be in main (argc=0, argv=0x7ffff7c397bb <__GI_raise+267>, env=) at perlmain.c:132

This happens since the introduction of Unicode property wildcards.

[Please do not change anything below this line]
Flags:
category=core
severity=medium
Site configuration information for perl 5.31.6:

Configured by dur-randir at Fri Nov 8 05:18:19 MSK 2019.

Summary of my perl5 (revision 5 version 31 subversion 6) configuration:
Commit id: 1462134
Platform:
osname=darwin
osvers=13.4.0
archname=darwin-2level
uname='darwin isengard.local 13.4.0 darwin kernel version 13.4.0: mon jan 11 18:17:34 pst 2016; root:xnu-2422.115.15~1release_x86_64 x86_64 '
config_args='-de -Dusedevel -DDEBUGGING'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler:
cc='cc'
ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9 -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -I/opt/local/include -DPERL_USE_SAFE_PUTENV'
optimize='-O3 -g'
cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9 -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -I/opt/local/include'
ccversion=''
gccversion='4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='cc'
ldflags =' -mmacosx-version-min=10.9 -fstack-protector -L/usr/local/lib -L/opt/local/lib'
libpth=/usr/local/lib /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib /usr/lib /opt/local/lib
libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc
perllibs=-lpthread -ldl -lm -lutil -lc
libc=
so=dylib
useshrplib=false
libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=bundle
d_dlsymun=undef
ccdlflags=' '
cccdlflags=' '
lddlflags=' -mmacosx-version-min=10.9 -bundle -undefined dynamic_lookup -L/usr/local/lib -L/opt/local/lib -fstack-protector'

@inc for perl 5.31.6:
lib
/usr/local/lib/perl5/site_perl/5.31.6/darwin-2level
/usr/local/lib/perl5/site_perl/5.31.6
/usr/local/lib/perl5/5.31.6/darwin-2level
/usr/local/lib/perl5/5.31.6

Environment for perl 5.31.6:
DYLD_LIBRARY_PATH (unset)
HOME=/Users/dur-randir
LANG=en_US.UTF-8
LANGUAGE (unset)
LC_CTYPE=en_US.UTF-8
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.26.0/bin:/opt/local/bin:/usr/texbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin
PERLBREW_HOME=/Users/dur-randir/.perlbrew
PERLBREW_MANPATH=/Users/dur-randir/perlbrew/perls/perl-5.26.0/man
PERLBREW_PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.26.0/bin
PERLBREW_PERL=perl-5.26.0
PERLBREW_ROOT=/Users/dur-randir/perlbrew
PERLBREW_SHELLRC_VERSION=0.86
PERLBREW_VERSION=0.86
PERL_BADLANG (unset)
SHELL=/opt/local/bin/zsh

[khwilliamson]

I can't reproduce this in current blead. I believe this was fixed by cd9d511

[dur-randir]

This build is from 5af38e4, which is later. Config args are -des -Dusedevel -DDEBUGGING -Dcc=afl-clang-fast -Doptimize=-std=c99 -O3 -funroll-loops -g.

[khwilliamson]

Before I try it, the text of your ticket says:

While fuzzing perl v5.31.5-213-g9bec17d7c built with afl and run
under libdislocator, I found the following program

0=~/\p{nv=}(?0)|\337ss|\337ss/
...
Configured by dur-randir at Fri Nov 8 05:18:19 MSK 2019.

Summary of my perl5 (revision 5 version 31 subversion 6) configuration:
Commit id: 1462134

I'm wondering if somehow this is completely wrong, and you meant to paste in something else.

[dur-randir]

I've actually re-tested this on 5af38e4 - I forgot to update the footer when I was posting this :( I have pre-generated templates for all those reports, and update them only irregularly.

[khwilliamson]

I don't have that compiler, but I tried it with clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
and it worked. And your results just don't make sense to me. The commit that should have fixed this caused the pattern compilation to abort when it can't find a terminator:

Unicode property wildcard not terminated "nv=" in regex; marked by <-- HERE in m/\p{nv=} <-- HERE (?0)|\337ss|\337ss/ at -e line 1.

That's the results I am getting. make_trie wouldn't be getting called at all, since a syntax error was found.

The results I'm getting are consistent with this, and the results you are getting look like they're from an earlier blead, before the catching of the subpattern terminator was fixed.

[dur-randir]

I've updated the ticked with correct escapes, now it contains proper number of backslashes.