(Mac)Debug ALL assertion fails when instancing embedded code before a named capture followed by named back reference

[AnFunctionArray]

Module: re

NOTE: Just seems to be a Mac only issue please post your version if you are reporting as a false positive.

I just tested on a Mac 12.1 virtual machine (in case my original installation was corrupt) and is still reproducible).

Description
Assertion failed when I use qw(Debug ALL) in my program (on my default built-in perl mac os 12.1).

(_svpvx) & SVt_MASK]), function my_regprop, file re_comp.c, line 20441.
zsh: abort      perl test.pl

Steps to Reproduce

use re qw(Debug ALL);

qr{(?{callfunc})(?<name>\g{absqualifsbasic}}

Expected behavior
Not assert fail.

Perl configuration

Summary of my perl5 (revision 5 version 30 subversion 3) configuration:
   
  Platform:
    osname=darwin
    osvers=21.0
    archname=darwin-thread-multi-2level
    uname='darwin bb-g9-pdb59.ta10.sd.apple.com 21.0 darwin kernel version 20.1.0: tue jun 29 22:08:19 pdt 2021; root:xnu-7195.41.8.100.7~1development_x86_64 x86_64 '
    config_args='-ds -e -Dprefix=/usr -Dccflags=-g  -pipe  -Dldflags= -Dman3ext=3pm -Duseithreads -Duseshrplib -Dinc_version_list=none -Dcc=cc'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags =' -g -pipe -fno-strict-aliasing -fstack-protector-strong -DPERL_USE_SAFE_PUTENV'
    optimize='-Os'
    cppflags='-g -pipe -fno-strict-aliasing -fstack-protector-strong'
    ccversion=''
    gccversion='Apple LLVM 13.0.0 (clang-1300.0.29.10) [+internal-os, ptrauth-isa=deployment-target-based]'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector-strong'
    libpth=/System/Volumes/Data/SWE/macOS/BuildRoots/5b2e67f8af/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.Internal.sdk/usr/local/lib /System/Volumes/Data/SWE/macOS/BuildRoots/5b2e67f8af/Applications/Xcode.app/Contents/Developer/Toolchains/OSX12.1.xctoolchain/usr/lib/clang/13.0.0/lib /System/Volumes/Data/SWE/macOS/BuildRoots/5b2e67f8af/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.Internal.sdk/usr/lib /System/Volumes/Data/SWE/macOS/BuildRoots/5b2e67f8af/Applications/Xcode.app/Contents/Developer/Toolchains/OSX12.1.xctoolchain/usr/lib /System/Volumes/Data/SWE/macOS/BuildRoots/5b2e67f8af/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib /usr/lib /usr/local/lib
    libs= 
    perllibs=
    libc=
    so=dylib
    useshrplib=true
    libperl=libperl.dylib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -bundle -undefined dynamic_lookup -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
    USE_THREAD_SAFE_LOCALE
  Locally applied patches:
    /Library/Perl/Updates/<version> comes before system perl directories
    installprivlib and installarchlib points to the Updates directory
  Built under darwin
  Compiled at Nov 13 2021 00:39:53
  @INC:
    /Library/Perl/5.30/darwin-thread-multi-2level
    /Library/Perl/5.30
    /Network/Library/Perl/5.30/darwin-thread-multi-2level
    /Network/Library/Perl/5.30
    /Library/Perl/Updates/5.30.3
    /System/Library/Perl/5.30/darwin-thread-multi-2level
    /System/Library/Perl/5.30
    /System/Library/Perl/Extras/5.30/darwin-thread-multi-2level
    /System/Library/Perl/Extras/5.30

[AnFunctionArray]

I closed since the example worked on online perl but maybe this is a macos exclusive issue.

[demerphq]

On Wed, 19 Jan 2022 at 01:38, 6a4h8 ***@***.***> wrote: Module: re *Description* Assertion failed when I use qw(Debug ALL) in my program. *Steps to Reproduce* use re qw(Debug ALL); qr{(?{callfunc})(?<name>\g{absqualifsbasic}} *Expected behavior* Not assert fail. *Perl configuration* Summary of my perl5 (revision 5 version 30 subversion 3) configuration: No assert fail in 5.28.1:

$ perl -Mre=Debug,ALL -e'qr{(?{callfunc})(?<name>\g{absqualifsbasic}}' Assembling pattern from 2 elements Compiling REx "(?{callfunc})(?<name>\g{absqualifsbasic}" Starting first pass (sizing) <(?{callfunc>...| 1| reg | | brnc | | piec | | atom <?{callfunc}>...| | reg <(?<name>\g{>...| 4| piec | | atom <?<name>\g{a>...| | reg <\g{absquali>...| 6| brnc | | piec | | atom Unmatched ( in regex; marked by <-- HERE in m/(?{callfunc})( <-- HERE ?<name>\g{absqualifsbasic}/ at -e line 1.

…

-- perl -Mre=debug -e "/just|another|perl|hacker/"

[AnFunctionArray]

On Wed, 19 Jan 2022 at 01:38, 6a4h8 @.***> wrote: Module: re Description Assertion failed when I use qw(Debug ALL) in my program. Steps to Reproduce use re qw(Debug ALL); qr{(?{callfunc})(?\g{absqualifsbasic}} Expected behavior Not assert fail. Perl configuration Summary of my perl5 (revision 5 version 30 subversion 3) configuration: No assert fail in 5.28.1:
$ perl -Mre=Debug,ALL -e'qr{(?{callfunc})(?\g{absqualifsbasic}}' Assembling pattern from 2 elements Compiling REx "(?{callfunc})(?\g{absqualifsbasic}" Starting first pass (sizing) <(?{callfunc>...| 1| reg | | brnc | | piec | | atom <?{callfunc}>...| | reg <(?\g{>...| 4| piec | | atom <?\g{a>...| | reg <\g{absquali>...| 6| brnc | | piec | | atom Unmatched ( in regex; marked by <-- HERE in m/(?{callfunc})( <-- HERE ?\g{absqualifsbasic}/ at -e line 1.
…
-- perl -Mre=debug -e "/just|another|perl|hacker/"

Did you try on a mac? what is your perl version?

[demerphq]

On Wed, 19 Jan 2022 at 06:15, 6a4h8 ***@***.***> wrote: On Wed, 19 Jan 2022 at 01:38, 6a4h8 *@*.***> wrote: Module: re *Description* Assertion failed when I use qw(Debug ALL) in my program. *Steps to Reproduce* use re qw(Debug ALL); qr{(?{callfunc})(?\g{absqualifsbasic}} *Expected behavior* Not assert fail. *Perl configuration* Summary of my perl5 (revision 5 version 30 subversion 3) configuration: No assert fail in 5.28.1: $ perl -Mre=Debug,ALL -e'qr{(?{callfunc})(?\g{absqualifsbasic}}' Assembling pattern from 2 elements Compiling REx "(?{callfunc})(?\g{absqualifsbasic}" Starting first pass (sizing) <(?{callfunc>...| 1| reg | | brnc | | piec | | atom <?{callfunc}>...| | reg <(?\g{>...| 4| piec | | atom <?\g{a>...| | reg <\g{absquali>...| 6| brnc | | piec | | atom Unmatched ( in regex; marked by <-- HERE in m/(?{callfunc})( <-- HERE ?\g{absqualifsbasic}/ at -e line 1. … <#m_-5255632312895881260_> -- perl -Mre=debug -e "/just|another|perl|hacker/" Did you try on a mac?

Nope. I have a linux box.

what is your perl version?

Stated above 5.28.1 yves.

[AnFunctionArray]

On Wed, 19 Jan 2022 at 06:15, 6a4h8 @.> wrote: On Wed, 19 Jan 2022 at 01:38, 6a4h8 @.> wrote: Module: re Description Assertion failed when I use qw(Debug ALL) in my program. Steps to Reproduce use re qw(Debug ALL); qr{(?{callfunc})(?\g{absqualifsbasic}} Expected behavior Not assert fail. Perl configuration Summary of my perl5 (revision 5 version 30 subversion 3) configuration: No assert fail in 5.28.1: $ perl -Mre=Debug,ALL -e'qr{(?{callfunc})(?\g{absqualifsbasic}}' Assembling pattern from 2 elements Compiling REx "(?{callfunc})(?\g{absqualifsbasic}" Starting first pass (sizing) <(?{callfunc>...| 1| reg | | brnc | | piec | | atom <?{callfunc}>...| | reg <(?\g{>...| 4| piec | | atom <?\g{a>...| | reg <\g{absquali>...| 6| brnc | | piec | | atom Unmatched ( in regex; marked by <-- HERE in m/(?{callfunc})( <-- HERE ?\g{absqualifsbasic}/ at -e line 1. … <#m_-5255632312895881260_> -- perl -Mre=debug -e "/just|another|perl|hacker/" Did you try on a mac?
Nope. I have a linux box.
what is your perl version?
Stated above 5.28.1 yves.

Well it seems to be an issue only with Mac - (tested on mac mini m1 with both bult-in version and blead and also on a vm inside the mac).

[choroba]

Fails with the latest blead perl on Linux:

$ blead perl -e'use re Debug=>"ALL";qr{(?{a})(?<b>\g{c})}'
Assembling pattern from 2 elements
Compiling REx "(?{a})(?<b>\g{c})"
Starting parse and generation
<(?{a})(?<b>>...|   1|  reg    
                |    |    brnc   
                |    |      piec   
                |    |        atom   
<?{a})(?<b>\>...|    |          reg    
<(?<b>\g{c})>   |   4|      piec   
                |    |        atom   
<?<b>\g{c})>    |    |          reg    
                |    |            Setting open paren #1 to 4
<\g{c})>        |   6|            brnc   
                |    |              piec   
                |    |                atom   
<)>             |   8|            tail~ OPEN1 'b' (4) -> REFN
                |    |            Setting close paren #1 to 8
                |  10|          lsbrperl5.35.9: re_comp.c:21443: my_regprop: Assertion `PL_valid_types_PVX[SvTYPE(_svpvx) & SVt_MASK]' failed.
/home/choroba/bin/blead: line 26: 31743 Aborted                 (core dumped) "${exec[0]}" "$@"

$ blead -v

This is perl 5, version 35, subversion 9 (v5.35.9 (v5.35.8-7-g8030c9e253)) built for x86_64-linux-thread-multi

[demerphq]

On Sat, 22 Jan 2022 at 21:27, E. Choroba ***@***.***> wrote: Fails with the latest blead perl on Linux: $blead -e'use re Debug=>"ALL";qr{(?{a})(?<b>\g{c}})' Assembling pattern from 2 elements Compiling REx "(?{a})(?<b>\g{c}" Starting parse and generation <(?{a})(?<b>>...| 1| reg | | brnc | | piec | | atom <?{a})(?<b>\>...| | reg <(?<b>\g{c}> | 4| piec | | atom <?<b>\g{c}> | | reg | | Setting open paren #1 to 4 <\g{c}> | 6| brnc | | piec | | atom <> | 8| tail~ OPEN1 'b' (4) -> REFN | | Setting close paren #1 to 8 | 10| lsbrperl5.35.9: re_comp.c:21443: my_regprop: Assertion `PL_valid_types_PVX[SvTYPE(_svpvx) & SVt_MASK]' failed. /home/choroba/bin/blead: line 26: 18460 Aborted (core dumped) "${exec[0]}" "$@" $ blead -v This is perl 5, version 35, subversion 9 (v5.35.9 (v5.35.8-7-g8030c9e253)) built for x86_64-linux-thread-multi (with 1 registered patch, see perl -V for more detail)

Thanks. This is broken by: commit 2a98b8c Author: Richard Leach ***@***.***> Date: Tue Nov 23 23:35:58 2021 +0000 newSVpvn_flags().. is more efficient than sv_2mortal(newSVpvn(..)) The same holds for newSVpvs* wrappers around newSVpvn* functions. It is from the op.c change. I think something about this patch is tickling a compiler bug as the code appears to be correct but it randomly produces different results. I would push a fix patch, but my config is out of date, ill try to do it later. Yves

…

-- perl -Mre=debug -e "/just|another|perl|hacker/"

[demerphq]

On Sun, 23 Jan 2022 at 08:42, demerphq ***@***.***> wrote: On Sat, 22 Jan 2022 at 21:27, E. Choroba ***@***.***> wrote: > > <> | 8| tail~ OPEN1 'b' (4) -> REFN > | | Setting close paren #1 to 8 > | 10| lsbrperl5.35.9: re_comp.c:21443: my_regprop: Assertion `PL_valid_types_PVX[SvTYPE(_svpvx) & SVt_MASK]' failed. > /home/choroba/bin/blead: line 26: 18460 Aborted (core dumped) "${exec[0]}" "$@" > > $ blead -v > > This is perl 5, version 35, subversion 9 (v5.35.9 (v5.35.8-7-g8030c9e253)) built for x86_64-linux-thread-multi > (with 1 registered patch, see perl -V for more detail) > > > Thanks. This is broken by:

Actually I recant. I will dig more. There is something heisenbug about this. If I run the test repeatedly it fails or pases randomly, which is making it hard to find the patch that caused this. cheers Yves