Invalid mapping patterns give confusing SyntaxErrors

[brandtbucher]

BPO	44368
Nosy	@lysnikolaou, @pablogsal, @miss-islington, @brandtbucher
PRs	bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords #26630 [3.10] bpo-44368: Ensure we don't raise incorrect custom syntax errors with soft keywords (GH-26630) #26631 bpo-44368: Improve syntax errors with invalid as pattern targets #26632 [3.10] bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632) #26792

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-06-09.19:47:35.477>
labels = ['interpreter-core', 'type-bug', '3.11']
title = 'Invalid mapping patterns give confusing SyntaxErrors'
updated_at = <Date 2021-06-18.21:16:02.181>
user = 'https://github.com/brandtbucher'

bugs.python.org fields:

activity = <Date 2021-06-18.21:16:02.181>
actor = 'pablogsal'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Parser']
creation = <Date 2021-06-09.19:47:35.477>
creator = 'brandtbucher'
dependencies = []
files = []
hgrepos = []
issue_num = 44368
keywords = ['patch']
message_count = 20.0
messages = ['395453', '395454', '395458', '395460', '395461', '395462', '395467', '395469', '395470', '395473', '395477', '395478', '395479', '395481', '395482', '395485', '395486', '395487', '395584', '396092']
nosy_count = 4.0
nosy_names = ['lys.nikolaou', 'pablogsal', 'miss-islington', 'brandtbucher']
pr_nums = ['26630', '26631', '26632', '26792']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44368'
versions = ['Python 3.11']

[brandtbucher]

Here are a few that I found. Not sure when they were introduced:

match ...:
case {**rest, "key": value}:
pass

match ...:
case {"first": first, **rest, "last": last}:
pass

match ...:
case {**_}:
pass

These all give the following error while parsing the second line:

File "<stdin>", line 1
match ...:
^^^^^^^^^
SyntaxError: invalid syntax. Perhaps you forgot a comma?

[brandtbucher]

Perhaps we need something like invalid_mapping_pattern or invalid_mapping_pattern_double_star rules?

[pablogsal]

Probably, otherwise is going to hurt because the syntax errors that you describe trigger when two names are place together, which in general is a missing comma.

The error message also doesn't say: "you are missing a comma" it says that the most typical reason is a missing comma, which is important to distinguish :)

[pablogsal]

The backtracking with the soft keyword may make this very annoying, by the way.

[pablogsal]

Oh, wait, I think I misunderstood the problem. The problem is that the parser is backtracking and identifying match as a name.

Indeed, the soft keyword is a pain :(

[pablogsal]

I think it actually will be very useful to explain that these cases are invalid in the Syntax error, which will also solve this problem.

This on the other hand shows a bigger problem: any generic syntax error that happens inside "match" will probably end identifying the keyword as a name, even if we don't have specific errors, the parser will point to it and complain about "match".

[pablogsal]

Oh, turns out I already added machinery to solved this but I was missing a piece!

[brandtbucher]

Wow, that was quite a roller-coaster ride. Thanks Pablo. :)

[brandtbucher]

I found a similar one, by the way (not related to mapping patterns):

>>> match ...:
...     case 42 as _:
...         pass

  File "<stdin>", line 2
    case 42 as _:
               ^
SyntaxError: expected ':'

Is this covered by your fix?