[ty] Prevent unbounded expansion of recursive union aliases - #26882
Draft
mtshiba wants to merge 1 commit into
Draft
[ty] Prevent unbounded expansion of recursive union aliases#26882mtshiba wants to merge 1 commit into
mtshiba wants to merge 1 commit into
Conversation
Typing conformance resultsNo changes detected ✅Current numbersThe percentage of diagnostics emitted that were expected errors held steady at 96.83%. The percentage of expected errors that received a diagnostic held steady at 91.74%. The number of fully passing files held steady at 99/133. |
Memory usage reportSummary
Significant changesClick to expand detailed breakdownsphinx
prefect
trio
flake8
|
|
Merging this PR will not alter performance
Comparing 
Footnotes
|
mtshiba
force-pushed
the
recursive-union-expansion
branch
3 times, most recently
from
July 16, 2026 20:18
38a3ca5 to
827bd05
Compare
mtshiba
added a commit
that referenced
this pull request
Jul 19, 2026
## Summary Fixes ~~astral-sh/ty#3452~~ I intended to fix it, but the fixes were separated into #26881, #26882 and #26898. This PR provides a foundation for fixes, but there are observable improvements on this PR alone. For example: ```py from ty_extensions import static_assert from ty_extensions._internal import is_subtype_of type Left[T] = tuple[Left[list[T]]] type Right[T] = tuple[Right[list[T]]] # TODO: Left[int] should be equivalent to (subtype of) Right[int] static_assert(not is_subtype_of(Left[int], Right[int])) ``` This will result in stack overflow in the current main. A recursive type like `type StableRecursiveList[T] = T | list[StableRecursiveList[T]]` can be checked without any problem even in the current main. This is because the specialization that appears in the recursive type on the right side is the same as the left side, so the recursion guard using simple type equality works. The problem here is that among recursive aliases, the specialization on the right-hand side grows with each expansion. Current recursion guards cannot notice such type alias reentrancy. Therefore, when considering generic recursive type aliases, two levels of protection must be applied, distinguishing between equivalence based on type definition identity and full type equivalence, including specialization. Applying a recursive guard that only considers the latter will fail to detect cases of growing specialization (this is exactly the problem exposed by MRE). On the other hand, if we guard by considering only the former, specialization will not be considered, so we will treat `GrowingList[int]` and `GrowingList[str]` as the same thing. With this PR, the `CycleDetector` will now return a `Cycle` state in addition to `Ready` and `Pending` as a result of a visit. If the type aliases, including specializations, are equivalent to types already seen during the visit, the detector will still return the fallback value as `Ready`. If the type aliases are the same but the specializations are different, return `Cycle` and ask the upstream relation checker to decide. In the case of `TypeRelationChecker`, when this is received, it executes `recursive_type_pair_fallback` to complete the determination. --- So, what should `recursive_type_pair_fallback` do? If we call `check_type_pair` during this process, we will enter recursion again, so we need to make it a finite process. As I thought about it, I realized that this problem was undecidable. There is no general subtyping algorithm for growing recursive type aliases. This is because they will have expressive power equal to or greater than that of context-free grammars [^1]. In other words, determining the subtype of two such recursive aliases is the same problem as determining the equivalence and inclusion of two CFGs that are known to be undecidable. ```py from typing import Literal, final from ty_extensions import static_assert from ty_extensions._internal import is_subtype_of @Final class End: pass type AAnd[Rest] = tuple[Literal["a"], Rest] type BAnd[Rest] = tuple[Literal["b"], Rest] # S -> ε | aSb # {a^n b^n | n >= 0} type S[Rest] = Rest | AAnd[S[BAnd[Rest]]] # T -> ε | aTb | aaTbb # {a^n b^n | n >= 0} type T[Rest] = Rest | AAnd[T[BAnd[Rest]]] | AAnd[AAnd[T[BAnd[BAnd[Rest]]]]] # S and T produce exactly the same language, but the type checker cannot tell that. static_assert(not is_subtype_of(S[End], T[End])) ``` Therefore, we have to give up on this kind of recursive alias typing at some point (FYI, mypy makes growing recursive aliases like this illegal in the first place; pyright allows them, but seems to just have a recursion depth limit). In this PR, it simply returns a conservative solution immediately when a growing pattern of recursive type alias is detected. In reality, we can extend the support a little more, but we'll leave that as future work and focus on fixes first. --- #26881 strengthens the recursion guard in `TypeTransformer` under this PR change. This directly fixes #3452. #26882 strengthens the recursion guard in `UnionBuilder` under this PR change. This will properly stop the expansion of recursive union aliases that would result in stack overflow in the current main. #26898 adds `RecursionGuard` that wraps `TypeCollector`. [^1]: To be more specific, the type argument of type alias can be considered as the stack memory of a pushdown automaton. If the generic type alias itself appears on the right side with a different specialization than the left side, it corresponds to being able to push additional information onto the stack along with the state transition. If it has only trivial specializations like the left side, it cannot be used as a stack, and its abilities are equivalent to a finite automaton. ## Test Plan new mdtest cases --------- Co-authored-by: Micha Reiser <micha@reiser.io>
mtshiba
force-pushed
the
recursive-union-expansion
branch
from
July 20, 2026 14:35
827bd05 to
aa852c5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #26503
Test Plan
mdtest updated