`panic: restartop in perl_run`

[AnFunctionArray]

Module: perl

Description

When I run "test" =~ m{(?{eval {exists $test[-1]{test}}})}sxx (fails all the time)

Steps to Reproduce
Run.

Expected behavior
Not panic.

Perl configuration

Summary of my perl5 (revision 5 version 35 subversion 12) configuration:
  Commit id: 99db5f9692dfa6466693dce901a6e805243181fc
  Platform:
    osname=darwin
    osvers=21.4.0
    archname=darwin-2level
    uname='darwin alexanders-macbook-air.local 21.4.0 darwin kernel version 21.4.0: fri mar 18 00:47:26 pdt 2022; root:xnu-8020.101.4~15release_arm64_t8101 arm64 '
    config_args='-des -Dusedevel'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=12.3 -fno-strict-aliasing -pipe -fstack-protector-strong -I/opt/local/include -DPERL_USE_SAFE_PUTENV'
    optimize='-O3'
    cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=12.3 -fno-strict-aliasing -pipe -fstack-protector-strong -I/opt/local/include'
    ccversion=''
    gccversion='Apple LLVM 13.1.6 (clang-1316.0.21.2.3)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=8
    longdblkind=0
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -mmacosx-version-min=12.3 -fstack-protector-strong -L/opt/local/lib'
    libpth=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.1.6/lib /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib /opt/local/lib /usr/lib
    libs= 
    perllibs=
    libc=
    so=dylib
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -mmacosx-version-min=12.3 -bundle -undefined dynamic_lookup -L/opt/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_DEVEL
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
  Built under darwin
  Compiled at Apr 21 2022 21:24:15
  @INC:
    /usr/local/lib/perl5/site_perl/5.35.12/darwin-2level
    /usr/local/lib/perl5/site_perl/5.35.12
    /usr/local/lib/perl5/5.35.12/darwin-2level
    /usr/local/lib/perl5/5.35.12

[jkeenan]

On 5/1/22 09:35, AnFunctionArray wrote: Module: perl *Description* Basically sometimes when I run my program I get |panic: restartop in perl_run| (1 in 5). *Steps to Reproduce* Run like 10 times (fast). *Expected behavior* Not panic. *Perl configuration* |Summary of my perl5 (revision 5 version 35 subversion 12) configuration: Commit id: 99db5f9

Have you observed this problem with production releases of perl (e.g., perl-5.34.1)?

[richardleach]

Using this tree: https://github.com/AnFunctionArray/regularc/tree/7287e18833598aae53a23c23141f60ee15a527e7 (clone recursively for examples)

Run the following test command:

perl ./regexes/parse.pl ./regularcbulk/tests/test.test.c

At least at first glance, ./regexes/parse.pl seems to have a lot going on, which is likely to deter volunteers from looking into this issue. Two things that would help:

• A much simplified test case that triggers the bug
• At least telling us how far through parse.pl, and ideally in which subroutine and/or module, the panic occurs

[AnFunctionArray]

I found it:

Try:

"test" =~ m{(?{eval {exists $test[-1]{test}}})}sxx

[hvds]

Confirmed with blead:

% ./miniperl -e '"test" =~ m{(?{eval {exists $test[-1]{test}}})}'
panic: restartop in perl_run
% 

Until some point between 5.22 and 5.30 it also then went on to assert in debugging builds:

% /opt/v5.22.1-d0/bin/perl -e '"test" =~ m{(?{eval {exists $test[-1]{test}}})}'
panic: restartop in perl_run
perl: perl.c:539: perl_destruct: Assertion `PL_scopestack_ix == 1' failed.
Aborted (core dumped)
% 

At 5.18 it doesn't complain; I'll try bisecting on that, but my first guess is that a change in 5.20 just exposed an even older bug.

It seems remotely possible this could be related to #19390, where we get a SEGV from:

  perl -wle 'use strict; use re "eval"; "" =~ m{(?{ eval q{ $x = 1 } })}'

[richardleach]

@hvds, is it relevant that something different happens with a couple of spaces in the mix? (i.e. is there a parsing bug?)

# perl -le '"test" =~ m{( ?{eval {exists $test[-1]{test}}} )}'
Modification of non-creatable array value attempted, subscript -1 at -e line 1.

[haarg]

With the spaces, it stops being an embedded code block. $test[-1]{test} just becomes an interpolated variable. Autovivifying an empty array with a -1 index gives that error.

$ perl -le '$test[-1]{test}'
Modification of non-creatable array value attempted, subscript -1 at -e line 1.

[hvds]

With the spaces, it stops being an embedded code block. $test[-1]{test} just becomes an interpolated variable. Autovivifying an empty array with a -1 index gives that error.

$ perl -le '$test[-1]{test}'
Modification of non-creatable array value attempted, subscript -1 at -e line 1.

Ah, I hadn't noticed that it would be an error, that suggests it's much more likely to be related to #19390.

[richardleach]

With the spaces, it stops being an embedded code block.

Thanks. I've never used re code blocks and wasn't sure whether whitespacing was allowed or not.

[hvds]

Manual bisect points to 1022336 as the point we first see the problem (strange, since the immediately preceding commit cdec98f looks more likely to be relevant; however reverting that one in blead doesn't fix the problem).

Unfortunately, simply reverting the main element of that commit as below causes miniperl to hang both in write_buildcustomize and in the test case - some following commits (e427370, 72621f8, c899ae2) built on the theme and there may have been other work later that assumed this work.

@iabyn does anything spring to mind?

--- a/op.c
+++ b/op.c
@@ -17573,7 +17573,7 @@ Perl_rpeep(pTHX_ OP *o)
         case OP_LINESEQ:
         case OP_SCOPE:
         nothin:
-            if (oldop) {
+            if (oldop && o->op_next) {
                 oldop->op_next = o->op_next;
                 o->op_opt = 0;
                 continue;

[iabyn]

On Tue, May 03, 2022 at 08:22:32AM -0700, Hugo van der Sanden wrote: Confirmed with blead: ``` % ./miniperl -e '"test" =~ m{(?{eval {exists $test[-1]{test}}})}' panic: restartop in perl_run %

It reduces further to p -e'"test" =~ m{(?{eval {die "boo!"}})}' (the $test[-1] was croaking about a non-modifiable array.) The bug was probably introduced by (or at least the ground prepared by) my jumbo re_eval patch that went into 5.18.0 and has probably been sensitive to the exact circumstances of a corrupted context stack ever since. I'm currently looking into it further.

…

-- The Enterprise successfully ferries an alien VIP from one place to another without serious incident. -- Things That Never Happen in "Star Trek" #7