Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 93 additions & 63 deletions crates/ty/docs/rules.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,35 @@ CustomerModel(id=1, name="Test")
CustomerModel()
```

### Metaclass with `kw_only_default=True`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test looks great. Nit: I think this section is to show basic usage, and arguments are tested below. I think we should move this down with the rest of the kw_only_default tests below, and just add an example with metaclass there. (I think the example here with decorator is probably redundant with the test we already have down there, and we don't need both?)


```py
from typing import dataclass_transform

@dataclass_transform(kw_only_default=True)
class ModelMeta(type):
pass

class WithMeta(metaclass=ModelMeta):
x: int

# error: [missing-argument]
# error: [too-many-positional-arguments]
WithMeta(3)

@dataclass_transform(kw_only_default=True)
def mydc():
return lambda func: func

@mydc
class WithDeco:
x: int

# error: [missing-argument]
# error: [too-many-positional-arguments]
WithDeco(3)
```

### Decorating a base class

```py
Expand Down Expand Up @@ -231,6 +260,28 @@ class CustomerModel:
c = CustomerModel(1, "Harry")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test immediately above here I thought should be fixed by logic you added in this PR, but then I realized why it isn't -- commented below.

```

When `kw_only_default=True` is set in the `dataclass_transform` decorator, all fields become
keyword-only by default and should not participate in positional field ordering checks:

```py
from typing import dataclass_transform

@dataclass_transform(kw_only_default=True)
def create_model(): ...
@create_model()
class ModelGood:
x: int = 1
y: str

@create_model()
class ModelAlsoGood:
x: int
y: str = "default"
z: float
```

Overwrite with field specifiers are not yet supported.

### `field_specifiers`

To do
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -110,14 +110,93 @@ reveal_type(D.__init__) # revealed: (self: D, x: int) -> None
```

If attributes without default values are declared after attributes with default values, a
`TypeError` will be raised at runtime. Ideally, we would emit a diagnostic in that case:
`TypeError` will be raised at runtime. We emit a diagnostic in that case:

```py
from dataclasses import field

@dataclass
class D:
x: int = 1
# TODO: this should be an error: field without default defined after field with default
# error: [dataclass-field-order] "Required field `y` cannot be defined after fields with default values"
y: str

@dataclass
class E:
x: int = field(default=3)
y: int = field(default_factory=lambda: 4)
# error: [dataclass-field-order]
z: str

import sys

@dataclass
class F:
x: int = 1
if sys.version_info > (3, 7):
# error: [dataclass-field-order]
y: str
```

Fields with `init=False` do not participate in the ordering check since they don't appear in
`__init__`:

```py
@dataclass
class GoodWithInitFalse:
x: int = 1
y: str = field(init=False)
z: float = 2.0

@dataclass
class BadWithInitFalse:
x: int = 1
y: str = field(init=False)
# error: [dataclass-field-order] "Required field `z` cannot be defined after fields with default values"
z: float
```

Keyword-only fields (using `kw_only=True`) also don't participate in the positional ordering check:

```toml
[environment]
python-version = "3.10"
```

```py
@dataclass
class GoodWithKwOnly:
x: int = 1
y: str = field(kw_only=True)
z: float = field(kw_only=True, default=2.0)

@dataclass
class BadWithKwOnly:
x: int = 1
y: str = field(kw_only=True)
# error: [dataclass-field-order] "Required field `z` cannot be defined after fields with default values"
z: float
```

Fields after a `KW_ONLY` sentinel are also keyword-only and don't participate in ordering checks:

```py
from dataclasses import KW_ONLY

@dataclass
class GoodWithKwOnlySentinel:
x: int = 1
_: KW_ONLY
y: str
z: float = 2.0

@dataclass
class BadWithKwOnlySentinel:
x: int = 1
# error: [dataclass-field-order] "Required field `y` cannot be defined after fields with default values"
y: str
_: KW_ONLY
z: float
```

Pure class attributes (`ClassVar`) are not included in the signature of `__init__`:
Expand Down Expand Up @@ -487,6 +566,23 @@ a = A(1, 2)
a = A(x=1, y=2)
```

No fields of a `kw_only=True` dataclass participate in field ordering checks.

```py
from dataclasses import dataclass, field

@dataclass(kw_only=True)
class KwOnlyClassGood:
x: int = 1
y: str

@dataclass(kw_only=True)
class KwOnlyClassAlsoGood:
x: int
y: str = "default"
z: float
```

The class-level parameter can be overridden per-field.

```py
Expand All @@ -500,6 +596,15 @@ class A:
A("hi")
```

```py
@dataclass(kw_only=True)
class KwOnlyClassWithPositionalField:
x: int = 1
y: str = field(kw_only=False, default="hello")
# error: [dataclass-field-order] "Required field `z` cannot be defined after fields with default values"
z: float = field(kw_only=False)
```

If some fields are `kw_only`, they should appear after all positional fields in the `__init__`
signature.

Expand Down
7 changes: 7 additions & 0 deletions crates/ty_python_semantic/src/semantic_index/definition.rs
Original file line number Diff line number Diff line change
Expand Up @@ -713,6 +713,13 @@ impl DefinitionKind<'_> {
}
}

pub(crate) fn as_annotated_assignment(&self) -> Option<&AnnotatedAssignmentDefinitionKind> {
match self {
DefinitionKind::AnnotatedAssignment(annassign) => Some(annassign),
_ => None,
}
}

/// Returns the [`TextRange`] of the definition target.
///
/// A definition target would mainly be the node representing the place being defined i.e.,
Expand Down
23 changes: 17 additions & 6 deletions crates/ty_python_semantic/src/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1408,13 +1408,16 @@ impl<'db> Type<'db> {

// Pretend that instances of `dataclasses.Field` are assignable to their default type.
// This allows field definitions like `name: str = field(default="")` in dataclasses
// to pass the assignability check of the inferred type to the declared type.
// to pass the assignability check of the inferred type to the declared type
// Note: a field without a default is assignable to the annotated type.
(Type::KnownInstance(KnownInstanceType::Field(field)), right)
if relation.is_assignability() =>
{
field
.default_type(db)
.has_relation_to_impl(db, right, relation, visitor)
.map_or(C::always_satisfiable(db), |ty| {
ty.has_relation_to_impl(db, right, relation, visitor)
})
}

(Type::TypedDict(_), _) | (_, Type::TypedDict(_)) => {
Expand Down Expand Up @@ -6696,7 +6699,9 @@ fn walk_known_instance_type<'db, V: visitor::TypeVisitor<'db> + ?Sized>(
// Nothing to visit
}
KnownInstanceType::Field(field) => {
visitor.visit_type(db, field.default_type(db));
if let Some(ty) = field.default_type(db) {
visitor.visit_type(db, ty);
}
}
}
}
Expand Down Expand Up @@ -6776,7 +6781,11 @@ impl<'db> KnownInstanceType<'db> {
KnownInstanceType::Deprecated(_) => f.write_str("warnings.deprecated"),
KnownInstanceType::Field(field) => {
f.write_str("dataclasses.Field[")?;
field.default_type(self.db).display(self.db).fmt(f)?;
if let Some(ty) = field.default_type(self.db) {
ty.display(self.db).fmt(f)?;
} else {
f.write_str("None")?;
}
f.write_str("]")
}
}
Expand Down Expand Up @@ -7116,7 +7125,8 @@ impl get_size2::GetSize for DeprecatedInstance<'_> {}
pub struct FieldInstance<'db> {
/// The type of the default value for this field. This is derived from the `default` or
/// `default_factory` arguments to `dataclasses.field()`.
pub default_type: Type<'db>,
/// `None` means no default value was provided.
pub default_type: Option<Type<'db>>,

/// Whether this field is part of the `__init__` signature, or not.
pub init: bool,
Expand All @@ -7132,7 +7142,8 @@ impl<'db> FieldInstance<'db> {
pub(crate) fn normalized_impl(self, db: &'db dyn Db, visitor: &NormalizedVisitor<'db>) -> Self {
FieldInstance::new(
db,
self.default_type(db).normalized_impl(db, visitor),
self.default_type(db)
.map(|ty| ty.normalized_impl(db, visitor)),
self.init(db),
self.kw_only(db),
)
Expand Down
14 changes: 9 additions & 5 deletions crates/ty_python_semantic/src/types/call/bind.rs
Original file line number Diff line number Diff line change
Expand Up @@ -940,11 +940,15 @@ impl<'db> Bindings<'db> {
overload.parameter_types()
{
let default_ty = match (default, default_factory) {
(Some(default_ty), _) => *default_ty,
(_, Some(default_factory_ty)) => default_factory_ty
.try_call(db, &CallArguments::none())
.map_or(Type::unknown(), |binding| binding.return_type(db)),
_ => Type::unknown(),
(Some(default_ty), _) => Some(*default_ty),
(_, Some(default_factory_ty)) => Some(
default_factory_ty
.try_call(db, &CallArguments::none())
.map_or(Type::unknown(), |binding| {
binding.return_type(db)
}),
),
_ => None,
};

let init = init
Expand Down
42 changes: 33 additions & 9 deletions crates/ty_python_semantic/src/types/class.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2378,17 +2378,25 @@ impl<'db> ClassLiteral<'db> {
.collect()
}

/// Returns a list of all annotated attributes defined in the body of this class. This is similar
/// to the `__annotations__` attribute at runtime, but also contains default values.
/// Returns a map of all annotated attributes defined in the body of this class.
/// This extends the `__annotations__` attribute at runtime by also including default values
/// and computed field properties.
///
/// For a class body like
/// ```py
/// @dataclass
/// @dataclass(kw_only=True)
/// class C:
/// x: int
/// y: str = "a"
/// y: str = "hello"
/// z: float = field(kw_only=False, default=1.0)
/// ```
/// we return a map `{"x": (int, None), "y": (str, Some(Literal["a"]))}`.
/// we return a map `{"x": Field, "y": Field, "z": Field}` where each `Field` contains
/// the annotated type, default value (if any), and field properties.
///
/// **Important**: The returned `Field` objects represent our full understanding of the fields,
/// including properties inherited from class-level dataclass parameters (like `kw_only=True`)
/// and dataclass-transform parameters (like `kw_only_default=True`). They do not represent
/// only what is explicitly specified in each field definition.
pub(super) fn own_fields(
self,
db: &'db dyn Db,
Expand Down Expand Up @@ -2440,7 +2448,6 @@ impl<'db> ClassLiteral<'db> {
let mut init = true;
let mut kw_only = None;
if let Some(Type::KnownInstance(KnownInstanceType::Field(field))) = default_ty {
default_ty = Some(field.default_type(db));
if self
.dataclass_params(db)
.map(|params| params.contains(DataclassParams::NO_FIELD_SPECIFIERS))
Expand All @@ -2449,7 +2456,9 @@ impl<'db> ClassLiteral<'db> {
// This happens when constructing a `dataclass` with a `dataclass_transform`
// without defining the `field_specifiers`, meaning it should ignore
// `dataclasses.field` and `dataclasses.Field`.
default_ty = Some(Type::unknown());
} else {
default_ty = field.default_type(db);
init = field.init(db);
kw_only = field.kw_only(db);
}
Expand All @@ -2468,9 +2477,24 @@ impl<'db> ClassLiteral<'db> {
kw_only_sentinel_field_seen = true;
}

// If no explicit kw_only setting and we've seen KW_ONLY sentinel, mark as keyword-only
if field.kw_only.is_none() && kw_only_sentinel_field_seen {
field.kw_only = Some(true);
// If no explicit field-level "kw_only" setting,
// we check for KW_ONLY sentinel or class-level settings
if field.kw_only.is_none() {
if kw_only_sentinel_field_seen {
// KW_ONLY sentinel makes subsequent fields keyword-only
field.kw_only = Some(true);
} else if self
.dataclass_params(db)
.is_some_and(|params| params.contains(DataclassParams::KW_ONLY))
|| self.try_metaclass(db).is_ok_and(|(_, transformer_params)| {
transformer_params.is_some_and(|params| {
params.contains(DataclassTransformerParams::KW_ONLY_DEFAULT)
})
})
{
// `@dataclass(kw_only=True)` or `@dataclass_transform(kw_only_default=True)`
field.kw_only = Some(true);
Comment on lines +2488 to +2496
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a test (I commented above) that I would expect this logic to fix, but it doesn't fix it. And I think I understand why. Currently DataclassParams is just a bitset, but this isn't really adequate when we have to compose it with DataclassTransformerParams. With the bitset representation, we can't distinguish "no kw_only argument" from kw_only=False -- they are both represented by the bit being off. I think we will need to change DataclassParams so that at least kw_only can be tri-valued. Because "no kw_only arg" should default to the dataclass-transform's kw_only_default value, but kw_only=False should potentially override it.

I think this doesn't need to be fixed in this PR, but it does impact the correctness of this code, so I think we should add a TODO comment here.

Suggested change
.is_some_and(|params| params.contains(DataclassParams::KW_ONLY))
|| self.try_metaclass(db).is_ok_and(|(_, transformer_params)| {
transformer_params.is_some_and(|params| {
params.contains(DataclassTransformerParams::KW_ONLY_DEFAULT)
})
})
{
// `@dataclass(kw_only=True)` or `@dataclass_transform(kw_only_default=True)`
field.kw_only = Some(true);
// TODO `kw_only=False` should override `kw_only_default=True`, this will require
// a tri-valued representation for `DataclassParams::KW_ONLY`
.is_some_and(|params| params.contains(DataclassParams::KW_ONLY))
|| self.try_metaclass(db).is_ok_and(|(_, transformer_params)| {
transformer_params.is_some_and(|params| {
params.contains(DataclassTransformerParams::KW_ONLY_DEFAULT)
})
})
{
// `@dataclass(kw_only=True)` or `@dataclass_transform(kw_only_default=True)`
field.kw_only = Some(true);

}
}

attributes.insert(symbol.name().clone(), field);
Expand Down
Loading
Loading