bpo-32873: Treat type variables and special typing forms as immutable by copy and pickle #6216
Conversation
|
@gvanrossum @serhiy-storchaka The beta 3 is scheduled for tomorrow, and I would like this fix to get in, could one of you please review this PR, it is quite small, but I believe important, in particular because it should restore (to the extent possible) the Sorry for short notice! |
| global TP, TPB, TPV # for pickle | ||
| TP = TypeVar('TP') | ||
| TPB = TypeVar('TPB', bound=int) | ||
| TPV = TypeVar('TPV', bytes, str) |
There was a problem hiding this comment.
I don't see TPB and TPV mentioned in the for-loop below?
| @@ -558,6 +565,7 @@ def __init__(self, name, *constraints, bound=None, | |||
| self.__bound__ = _type_check(bound, "Bound must be a type.") | |||
| else: | |||
| self.__bound__ = None | |||
| self._def_mod = sys._getframe(1).f_globals['__name__'] # for pickling | |||
There was a problem hiding this comment.
If the type variable is defined in some non-global scope (even inside a class) this won't work. I think that's fine (we need to support those type variables but they don't need to be picklable) but I wonder if this deserves at least a mention in the docstring.
I also notice that for the same reason a type variable defined in some non-global scope cannot be copy()'s, but it can be deepcopy()'d. That seems a little weird.
|
(The announced cutoff for 3.7.0b3 is in about 18 hours so there's still time.) |
|
@gvanrossum @ned-deily OK, great! I will fix this in 2-3 hours. |
|
Thanks @ilevkivskyi for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7. |
|
GH-6264 is a backport of this pull request to the 3.7 branch. |
… by copy and pickle (pythonGH-6216) This also fixes python/typingGH-512 This also fixes python/typingGH-511 As was discussed in both issues, some typing forms deserve to be treated as immutable by copy and pickle modules, so that: * copy(X) is X * deepcopy(X) is X * loads(dumps(X)) is X GH- pickled by reference This PR adds such behaviour to: * Type variables * Special forms like Union, Any, ClassVar * Unsubscripted generic aliases to containers like List, Mapping, Iterable This not only resolves inconsistencies mentioned in the issues, but also improves backwards compatibility with previous versions of Python (including 3.6). Note that this requires some dances with __module__ for type variables (similar to NamedTuple) because the class TypeVar itself is define in typing, while type variables should get module where they were defined. https://bugs.python.org/issue32873 (cherry picked from commit 8349403) Co-authored-by: Ivan Levkivskyi <levkivskyi@gmail.com>
… by copy and pickle (GH-6216) This also fixes python/typingGH-512 This also fixes python/typingGH-511 As was discussed in both issues, some typing forms deserve to be treated as immutable by copy and pickle modules, so that: * copy(X) is X * deepcopy(X) is X * loads(dumps(X)) is X GH- pickled by reference This PR adds such behaviour to: * Type variables * Special forms like Union, Any, ClassVar * Unsubscripted generic aliases to containers like List, Mapping, Iterable This not only resolves inconsistencies mentioned in the issues, but also improves backwards compatibility with previous versions of Python (including 3.6). Note that this requires some dances with __module__ for type variables (similar to NamedTuple) because the class TypeVar itself is define in typing, while type variables should get module where they were defined. https://bugs.python.org/issue32873 (cherry picked from commit 8349403) Co-authored-by: Ivan Levkivskyi <levkivskyi@gmail.com>
This also fixes python/typing#512
This also fixes python/typing#511
As was discussed in both issues, some typing forms deserve to be treated as immutable by
copyandpicklemodules, so that:This PR adds such behaviour to
Union,Any,ClassVarList,Mapping,IterableThis not only resolves inconsistencies mentioned in the issues, but also improves backwards compatibility with previous versions of Python (including 3.6).
Note that this requires some dances with
__module__for type variables (similar toNamedTuple)because the class
TypeVaritself is define intyping, while type variables should get module where they were defined.https://bugs.python.org/issue32873