Allow using typeddict for more precise typing of kwargs

[franekmagiera]

Description

Closes #4441

Added Expand special form to mypy_extensions python/mypy_extensions#22
Changes happen during semantic analysis. Firstly, semantic analyzer determines if kwargs are typed using Expand. If they are, type analyzer's visit_callable_type expands the keyword arguments using the TypedDict provided in Expand.
Semantic analyzer adds the kwargs variable to local symbol table that has explicit TypedDict type provided in Expand. Then function definition is modified.

Test Plan

Added a new file test-data/unit/check-expand.test that tests the changes.

I think overall it's a reasonable solution, but I have some doubts about the Expand special form in mypy_extensions, I use _SpecialForm from typing module and this may not work on all python versions. I will appreciate any hints on how to resolve this.

Another thing - should a note appear when **kwargs are typed using a TypedDict with total=False to encourage the use of kwargs.get(...) instead of kwargs[...]?

[franekmagiera]

@JukkaL @ilevkivskyi @hauntsaninja @ethanhs @JelleZijlstra what do you think about this?

[franekmagiera]

@JukkaL @ilevkivskyi @hauntsaninja @ethanhs @JelleZijlstra any thoughts on this one? Is this something you would be interested in merging? I am wondering if this approach makes sense given that there are some plans on improving typing for tensors in the future (not sure if those would be related with this issue)?

[hauntsaninja]

This is a useful feature, so thank you for the PR and driving this forward. However, (and speaking only for myself here), adding new constructs to the typing system is probably best done by starting a thread on the typing-sig mailing list, so that other type checkers and users of static typing can weigh in. That (plus writing a PEP) would also be the path to getting this into typing_extensions and Python 3.11's typing module.
I don't think anyone tagged is working closely on PEP 646, so typing-sig is probably the best way to get feedback on whether e.g. PEP 646's Unpack can be used for this purpose, or if there are other overlaps with variadic generics plans.

[rattrayalex]

Looks like the author of this PR has a draft PEP for this here: https://github.com/franekmagiera/peps/blob/more-precise-kwargs-typing/pep-9999.rst Very exciting, wish you luck @franekmagiera !

[franekmagiera]

@rattrayalex thanks, still need to describe the runtime behavior in more detail, recently it's been hard to find time but I hope to have the draft ready soon

[ilevkivskyi]

@franekmagiera Are you still working on this? Many things seem to have changed: the PEP specifies Unpack[] for this and mypy now has an UnpackType internally. Also it seems to me your approach will not work in fine-grained incremental mode.

[franekmagiera]

@ilevkivskyi Not directly on this PR but I'm working on a PEP draft:

• PEP 692: Introduce proposed enhancements and expand Motivation section peps#2732
• https://peps.python.org/pep-0692/

I think this PR could be closed, I agree that this approach will not work fully.

[ilevkivskyi]

OK, I see. Please feel free to propose another PR according to latest version of the PEP. Alternatively I can try implementing this myself.