pre-3.12 python_versions do not complain about f-strings syntax, but they should

[wyattscarpenter]

Bug Report

pre-3.12 python_versions do not complain about f-strings syntax, but they should, as those pythons could not parse those f-strings.

To Reproduce

Gist URL: https://gist.github.com/mypy-play/4606f2bda8d271532cac71318aa3eb84
Playground URL: https://mypy-play.net/?mypy=latest&python=3.9&flags=strict&gist=4606f2bda8d271532cac71318aa3eb84

ignored_codes = ["1","2"]

f'Error {', '.join(ignored_codes)}'

Expected Behavior

[Syntax] error

Actual Behavior

Success!

[sterliakov]

We do report syntax errors that result in new AST nodes that don't yet exist on the target python_version, but do not add any extra effort to check the rest.

Detecting this quoting issue, for example, will be really non-trivial - mypy works on AST, not on CST, so details like a quote type are immediately discarded during parsing. Mypy uses python parser matching the runtime version: if you're running python3.10 -m mypy, you get a syntax error, if you're running python3.13 -mmypy, you get no error. We do not bundle a dedicated parser for every python version (and that would be challenging). If you want mypy to detect all such problems, use separate venvs with correct interpreters instead of --python-version flag. That flag is mostly intended to resolve version guards in typeshed and your own code, not to check the syntactic validity.

This is not the same thing as, say, checking for a match/case on py3.9: match results in a real AST node, and we can trivially check that it should only exist with target python_version 3.10 or greater.

I'd say the rest is a job for the linter, not for type checker. Ruff has a corresponding rule for a while (astral-sh/ruff#6591), so this is an area mypy may completely ignore - linters can do better, they already parse the code into CST. Finally, any syntax error will be detected even with a very low test coverage: it's enough to import the file to make the test die hard if the syntax is not supported. And hopefully all projects relying on mypy do have at least basic tests and run them on all supported versions (or at least lowest + highest)...

[wyattscarpenter]

Hmm, well, maybe the new parser #19776 will be able to accomplish this easily. But I wouldn't be upset if this were closed as too difficult or out of scope.