[ty] Introduce a representation for the top/bottom materialization of an invariant generic

[JelleZijlstra]

Part of #994. This adds a new field to the Specialization struct to record when we're dealing with the top or bottom materialization of an invariant generic. It also implements subtyping and assignability for these objects.

Next planned steps after this is done are to implement other operations on top/bottom materializations; probably attribute access is an important one.

[github-actions]

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

[github-actions]

mypy_primer results

No ecosystem changes detected ✅
No memory usage changes detected ✅

[JelleZijlstra]

This feels not great and I tried to implement it in apply_specialization instead, but couldn't get that to work for a reason I don't fully understand.

[carljm]

I suspect the reason it didn't work is because apply_specialization is not called recursively, only apply_type_mapping_impl is? I think this would be difficult to change, so I suspect this is the way it needs to be done.

Maybe this would feel clearer if instead there were a top-level conditional here "are we applying a specialization?" (or it could be "are we applying a specialization with a materialization?" which I think is less conceptually clear but practically equivalent), and then bifurcate into the previous simple logic (if not) or the more complex logic accounting for specializations if so. (The latter could even be extracted into its own method.) This would duplicate the core "map over types" -- not really sure if it would feel like an improvement without trying it.

[JelleZijlstra]

What I tried was this:

--- a/crates/ty_python_semantic/src/types.rs
+++ b/crates/ty_python_semantic/src/types.rs
@@ -5971,7 +5971,11 @@ impl<'db> Type<'db> {
         db: &'db dyn Db,
         specialization: Specialization<'db>,
     ) -> Type<'db> {
-        self.apply_type_mapping(db, &TypeMapping::Specialization(specialization))
+        let new_specialization = self.apply_type_mapping(db, &TypeMapping::Specialization(specialization));
+        match specialization.specialization_type(db) {
+            None => new_specialization,
+            Some(materialization_type) => new_specialization.materialize(db, materialization_type)
+        }
     }
 
     fn apply_type_mapping<'a>(

Which should apply recursively since materialize is recursive.

[carljm]

Ah, I see, interesting. Yes, I see how that would be a lot nicer if it works; would eliminate the significant redundancy we currently have between apply_type_mapping_impl and materialize implementations.

It's not immediately clear to me why that wouldn't work. I think it's worth putting a TODO here to investigate this further, but it doesn't have to block this PR. What was the failure symptom? Was it widespread or just one or two specific cases?

If you have the failing version and want to push it up as a draft stacked PR, I can take a look and see if anything is apparent.

[JelleZijlstra]

#20121

[carljm]

It has something to do with how bases are specialized. Given this test code:

def _(y: Top[InvariantChild[Any]]) -> None:
    reveal_type(y.__mro__)

In the working branch, this reveals tuple[<class 'Top[InvariantChild[Any]]'>, <class 'CovariantBase[object]'>, typing.Generic, <class 'object'>], which is expected: the covariant base class materializes the typevar to object.

In the non-working branch, this reveals tuple[<class 'Top[InvariantChild[Any]]'>, <class 'CovariantBase[object]'>, typing.Generic, <class 'object'>, <class 'CovariantBase[Any]'>] -- for some reason we have both CovariantBase[object] and CovariantBase[Any] still in the MRO.

[carljm]

I'm out of time to look into this for now; I think we should go ahead and land this with a TODO comment. I'll add the comment and then merge.

[carljm]

Fantastic!