Move const folding to the peephole optimizer

[Eclips4]

Feature or enhancement

Proposal:

For additional context see #126830 (comment).

Flow graph optimizer has more information and can do better job.

The problem is that we need to convert from UNARY_OP(-, CONST(1)) to CONST(-1), still before the code generation phase, because this leads to a few problems, one of which is shown below.

x = 1

match x:
    case -0:
        y = 0

eclips4@nixos ~/p/p/cpython (remove-ast-optimizer)> ./python example.py
  File "/home/eclips4/programming/programming-languages/cpython/example.py", line 4
    case -0:
         ^^
SyntaxError: patterns may only match literals and attribute lookups

cc @markshannon

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• gh-126835: Disable tuple folding in the AST optimizer #128802
• gh-126835: Move optimization of constant sequence creation from codegen to CFG #129426
• gh-126835: Move constant unaryop & binop folding to CFG #129550
• gh-126835: Move constant subscript folding to CFG #129568
• gh-126835: Fix optimize_if_const_subscr refleaks #129634
• gh-126835: make CFG optimizer skip over NOP's when looking for const sequence construction #129703
• gh-126835: Fold unary & binary complex constant expressions during code generation. #129963
• gh-126835: Remove unused docstring const #130016
• gh-126835: Move const list & set folding in for_iter & contains from ast_opt.c to flowgraph.c #130032
• gh-126835: Set location for noped out instructions after constant folding in CFG. #130109
• gh-126835: Move constant tuple folding to CFG #130769
• gh-126835: Avoid creating unnecessary tuple when looking for constant sequence during constant folding #131054
• gh-126835: Refine constant folding tests in test_peepholer.py::TestTranforms #131826
• gh-126835: Rename AST optimization related stuff after moving const folding to the peephole optimizier #131830

[Eclips4]

We had a discussion with @tomasr8 and decided that splitting this into parts would be good. Initially, I tried to do it in one PR, but realized that it would be impossible to review, as it would do a lot of unrelated things.

To begin, we decided to start by moving tuple folding from AST optimizer to CFG.

Problem: Currently compiler emits a warning if is/is not operators are used with constants¹ (because it can evaluate them to True which is an implementation detail of CPython, other implementations might evaluate const is const to False).
So, the compiler (especially the codegen part) assumes that tuples are already folded, and has (in AST terms) Constant_kind. This breaks when we fold tuples later and they have a Tuple_kind at the codegen phase.

Possible solution: Perform this in a CFG after folding. This will be slower (I have no information on how slower it will be, but I assume it will) than performing this in codegen phase.

Footnotes

1. https://github.com/python/cpython/blob/2a66dd33dfc0b845042da9bb54aaa4e890733f54/Python/codegen.c#L1678-L1691 ↩

[WolframAlph]

Is there a plan what parts/in what order should be moved from ast optimizer to cfg? Does it make sense to create sub issues to make it more granular (instead of just PRs linked to this issue)? This issue Remove the AST optimizer seems too general and broad IMO.

[tomasr8]

I'm currently looking into moving unaryop/binop folding (though I'm at Fosdem atm so It might take a bit longer) Anything else is up for grabs I'd say, unless Kirill is working on something currently?

[WolframAlph]

I am looking into binop folding as well (almost done). Do you want to pass it or should I take something else?

[WolframAlph]

Problem with binop I discovered is that after moving it to cfg, we, for instance, run into problem with case matching which @Eclips4 mentioned in issue description. For example:

match x:
    case 0 + 0j:
        pass

raises

File "/Users/yyanchii/Desktop/cpython/internal/t.py", line 2
    case 0 + 0j:
         ^^^^^^
SyntaxError: patterns may only match literals and attribute lookups

It is raised from:

cpython/Python/codegen.c

Lines 6022 to 6025 in 89fe067

	if (!MATCH_VALUE_EXPR(value)) {
	const char *e = "patterns may only match literals and attribute lookups";
	return _PyCompile_Error(c, LOC(p), e);
	}

So at codegen stage, it already expects (in this case) binops to be folded. @tomasr8 did you run into this already? We would need to add more logic in here which doesn't look good to me. On the other hand, how would we move binop folding to CFG then?