[ty] Generate the top and bottom materialization of a type

[dhruvmanila]

Summary

This is to support #18607.

This PR adds support for generating the top materialization (or upper bound materialization) and the bottom materialization (or lower bound materialization) of a type. This is the most general and the most specific form of the type which is fully static, respectively.

More concretely, T', the top materialization of T, is the type T with all occurrences
of dynamic type (Any, Unknown, @Todo) replaced as follows:

• In covariant position, it's replaced with object
• In contravariant position, it's replaced with Never
• In invariant position, it's replaced with an unresolved type variable

(For an invariant position, it should actually be replaced with an existential type, but this is not currently representable in our type system, so we use an unresolved type variable for now instead.)

The bottom materialization is implemented in the same way, except we start out in "contravariant" position.

Test Plan

Add test cases for various types.

[github-actions]

mypy_primer results

No ecosystem changes detected ✅

[carljm]

This is looking good! I think the main thing is we need to ensure nesting is right, by always flipping the polarity of the "outer" variance when we hit a contravariant context, not hardcoding variance.

Another observation here is that the implementation of top_materialization and bottom_materialization should be identical -- the only difference is whether we start with a variance of "Covariant" (top materialization) or start with a variance of "Contravariant" (bottom materialization). It might make sense to rename the method added in the current PR to just materialize, and then have Type::top_materialization and Type::bottom_materialization just be thin wrappers that call materialize with the right initial variance.

[dhruvmanila]

Post review changes

• Add top_materialization, bottom_materialization, and a common materialize methods on Type
• Add TypeVarVariance::flip method that flips the polarity of the variance
• Apply the flip in negation type and callable parameter type
• Update SubclassOf to use variance and return Never for when contravariant and type for other variance
• Expanded the test suite to check bottom_materialization as well
• Add property tests, it's failing and I've a couple of questions regarding that I intend to ask in our 1:1

Edit:

• Change SubclassOf to use T: type when it's used in an invariant position

[carljm]

Looks great!

[jelle-openai]

Just snooping around here but is this missing a Type::SubclassOf in the return value?

[AlexWaygood]

nah, the function here delegates to SubclassOfType::materialize(), which does work to figure out which Type variant it needs to return (it won't always be a Type::SubclassOf type due to some eager normalizations we do):

ruff/crates/ty_python_semantic/src/types/subclass_of.rs

Lines 80 to 104 in a2cd6df

	pub(super) fn materialize(self, db: &'db dyn Db, variance: TypeVarVariance) -> Type<'db> {
	match self.subclass_of {
	SubclassOfInner::Dynamic(_) => match variance {
	TypeVarVariance::Covariant => KnownClass::Type.to_instance(db),
	TypeVarVariance::Contravariant => Type::Never,
	TypeVarVariance::Invariant => {
	// We need to materialize this to `type[T]` but that isn't representable so
	// we instead use a type variable with an upper bound of `type`.
	Type::TypeVar(TypeVarInstance::new(
	db,
	Name::new_static("T_all"),
	None,
	Some(TypeVarBoundOrConstraints::UpperBound(
	KnownClass::Type.to_instance(db),
	)),
	variance,
	None,
	TypeVarKind::Pep695,
	))
	}
	TypeVarVariance::Bivariant => unreachable!(),
	},
	SubclassOfInner::Class(_) => Type::SubclassOf(self),
	}
	}

[carljm]

No, but this is subtle and probably should have a comment. SubclassOfType::materialize returns a Type for the overall materialization of the subclass-of type, not just for the inner type. The reason for this is that we don't currently have a direct representation for type[T] (that is, subclass-of a typevar), but the materialization of type[Any] in an invariant position should be type[T] where T has no upper bound or constraints, and it turns out that this is equivalent to just T' where T': type -- so that's how we represent this type[T] we need to synthesize.

I think I did just spot another bug in looking at SubclassOfType::materialize, though. That method currently acts as if type[...] itself were bivariant (imposed no variance constraints). But in fact it is covariant. So both the Contravariant and Invariant branches there should be returning the type variable. EDIT: never mind, this is wrong -- I was thinking in terms of collecting constraints in variance inference, but here we are just flipping variance, not collecting constraints. So type[] being covariant just means it doesn't change the outer variance context.