Allow \Q\E inside counted quantifier

[NWilson]

Philip and David Wahlstedt noted a difference in behaviour between PCRE2 and Perl, in the handling of x{\Q1\E,2}. Potentially there are other similar cases where Perl allows a \Q, or a naked \E, and PCRE2 does not.

https://groups.google.com/d/msgid/pcre2-dev/79a9d274-c7bc-4cbb-8257-dc44814d90f6n%40googlegroups.com?utm_medium=email&utm_source=footer

[NWilson]

Philip says in the message above that, ideally, the behaviour in PCRE2 should be fixed.

I have read the section, "Gory details of parsing quoted constructs" in perlop, and it's clear that the PCRE2 system deviates a certain amount from Perl.

Perl allows empty \Q\E literally anywhere inside the regex, even (\Q\E?\Q\E:foo) (for (?:foo)). Secondly, as noted, there are some syntax features that use the characters A-Za-z0-9_ (which are the ones left untouched by quotemeta), so that z{\Q123\E} and (?\Qs\E) and \p{\QL\E} are accepted (for z{123}, (?s), and \p{L}).

Our entire pre-parser is written to read from a buffer of characters (the pattern) and so allowing \Q\E at every arbitrary point within the current single-pass parse from SPTR to METAs would be very intrusive.

It essentially requires an extra pass to be introduced, to match Perl: a quick linear scan over the pattern, hunting for \Q. If none found, then use the pattern as-is. Otherwise, perform the "double-quotish backslash interpolation" on the pattern into a temporary buffer, to strip \Q...\E. This will need to also maintain a map of character offsets (urgh). Then, parse from the temporary buffer. Before freeing it, un-map the erroroffset.

As a side-effect, we could also implement \L, \l, \U, \u, \F, just for fun.

The benefit would be removing all the intrusive "inescq" logic from pattern parsing. It's even special-cased in several places, such as skipping empty \Q\E when a conditional assertion is expected. That could all go away.

We could also fix the discrepancy in the documentation:

    Pattern            PCRE2 matches     Perl matches

    \QA\B\E            A\B               A\B
    \Q\\E              \                 \\E

(The explanation for this is the "double-quotish backslash interpolation" does process backslashes within \Q...\E. It processes them, but passes through sequences like \\ and \x00 for handling by the later regex parser.)

It also seems that \Q stacks in Perl. It appears that "\Q\,\Q\,\E\," compiles to quotemeta('\,' . quotemeta('\,') . '\,') so that the quoting is doubled up. Needless to say, this is horrible, and can be used to produce an exponentially-doubling size in the input pattern, just by stacking several \Q. That is not a nice Perl feature.