Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Final names and attributes #5522

Merged
merged 58 commits into from
Sep 11, 2018
Merged
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
c6125f1
Start adding tests
Aug 19, 2018
8f2bee6
Add some flags
Aug 19, 2018
61707ab
Add semanal logic for (not really) variables
Aug 20, 2018
75ccad3
Add checks for re-assignment; self and other fixes; work on tests
Aug 20, 2018
84ac046
Tests for invalid declarations
Aug 20, 2018
395cbd5
Move the logic for MemberExpr to checkmember.py; more tests
Aug 20, 2018
67c76bf
Implement special access on class objects
Aug 20, 2018
f6b173e
Fix class re-assignment; more tests
Aug 20, 2018
5053902
Create new Var with bare Final on self; search all bases for final; t…
Aug 20, 2018
8a4ed37
Multiple inheritance; fix lint
Aug 20, 2018
c6a2d99
Start writing docs
Aug 20, 2018
0ca2770
Final classes and methods (+ some tests)
Aug 21, 2018
aa3ab4b
Minor fixes; more overriding tests
Aug 21, 2018
ad42674
An overload fix; more tests
Aug 21, 2018
a106d08
Tweak error messages; move error logic to messages.py
Aug 21, 2018
37c4bb6
Add protocol tests
Aug 21, 2018
b6531d3
Add some comments and docstrings; some reorg
Aug 21, 2018
8366d3f
Add more docs
Aug 21, 2018
312cbc7
Minor fixes in docs
Aug 21, 2018
ec15a63
Some fine-grained stuff; add diff tests
Aug 22, 2018
c8a8d9b
Clarify constant re-export; add few tests
Aug 22, 2018
157a936
Add more incremental tests (coarse and fine)
Aug 22, 2018
eb945c9
Tweak some targets in fine-grained tests
Aug 22, 2018
58e4243
Only keep basic tests in coarse-grained
Aug 22, 2018
8602c61
Store final value for simple literals
Aug 22, 2018
7e05f7d
Prohibit augmented assignments and final within loops
Aug 23, 2018
fa7f8d5
Address CR
ilevkivskyi Aug 26, 2018
8d2973c
Fix alias on newer Python versions
ilevkivskyi Aug 26, 2018
430660e
Merge branch 'master' into final
ilevkivskyi Aug 29, 2018
11a157e
Move final docs to a separate file
Aug 31, 2018
b5dca3b
Add section header
Aug 31, 2018
1912fd8
Fix the rest of comments about docs
Aug 31, 2018
235b28f
Prohibit overriding writeable with final
Aug 31, 2018
86e883a
Fix some tests
ilevkivskyi Sep 1, 2018
aa8b436
Allow deferred final definition in __init__
ilevkivskyi Sep 1, 2018
95da7ce
Fix self-check
ilevkivskyi Sep 1, 2018
b50f3de
Update new Var final flags
ilevkivskyi Sep 1, 2018
2f74d87
Simplify/shorten some error messages
ilevkivskyi Sep 1, 2018
f9f1182
Merge remote-tracking branch 'upstream/master' into final
ilevkivskyi Sep 1, 2018
0364868
Further simplify error messages; use one-line errors instead of notes
ilevkivskyi Sep 1, 2018
632def5
Merge remote-tracking branch 'upstream/master' into final
ilevkivskyi Sep 1, 2018
cc8ce09
Merge branch 'master' into final
ilevkivskyi Sep 3, 2018
35bf8cf
Merge branch 'master' into final
ilevkivskyi Sep 5, 2018
53720e3
Remove ClassVar discussion
ilevkivskyi Sep 6, 2018
953119d
Update docs
ilevkivskyi Sep 6, 2018
ed009ee
Minor fixes in code
ilevkivskyi Sep 7, 2018
b8c67aa
Stricter rules for final in overloads; update tests
ilevkivskyi Sep 7, 2018
c9abd9d
Minor fixes
ilevkivskyi Sep 7, 2018
41b996f
Address some CR
ilevkivskyi Sep 11, 2018
7a85285
More CR; update tests
ilevkivskyi Sep 11, 2018
b37f35f
Prohibit bare final without r.h.s.
ilevkivskyi Sep 11, 2018
f521723
Merge remote-tracking branch 'upstream/master' into final
ilevkivskyi Sep 11, 2018
66fdbc8
Minor fixes
ilevkivskyi Sep 11, 2018
a76da46
Kill the doc part
ilevkivskyi Sep 11, 2018
aefd17c
Add test case for delayed definition in a subclass
Sep 11, 2018
309e1a5
Show less errors; add a test
Sep 11, 2018
98270fc
Fix self-check
Sep 11, 2018
8b52fda
Fix grammar
Sep 11, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
285 changes: 285 additions & 0 deletions docs/source/final_attrs.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,285 @@
Final names, methods and classes
================================

You can declare a variable or attribute as final, which means that the variable
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In think that the section structure could be improved by doing it like this:

= Final names, methods, and classes

<short explanation of final names (variables/attributes), methods and classes, ~one paragraph>

== Final names

<all discussion specific to Final>

== Final methods

<all discussion specific to @final when used with methods>

== Final classes

<all discussion specific to @final when used with classes>

must not be assigned a new value after initialization. This is often useful for
module and class level constants as a way to prevent unintended modification.
Mypy will prevent further assignments to final names in type-checked code:

.. code-block:: python

from typing_extensions import Final

RATE: Final = 3000
class Base:
DEFAULT_ID: Final = 0

# 1000 lines later

RATE = 300 # Error: can't assign to final attribute
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace can't -> cannot if you do the same change elsewhere.

Base.DEFAULT_ID = 1 # Error: can't override a final attribute
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here also can't -> cannot (and elsewhere in messages).


Another use case for final attributes is where a user wants to protect certain
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style nit: We don't use "a user". Prefer "you" instead, but in this case it can just be left out. For example: "Another use case for final attributes is to protect certain attributes form being overridden in a subclass".

instance attributes from overriding in a subclass:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this can also be used for class attributes. Maybe just leave out "instance"?


.. code-block:: python

import uuid
from typing_extensions import Final

class Snowflake:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example feels a bit too clever. I'd prefer something super straightforward that doesn't have these issues:

  • Snowflake would be an awkward name in real code. I think it's better to use more obviously dummy names such as Foo or C instead, but it would be even better to have somewhat realistic names.
  • The subclass doesn't actually override id properly since __init__ is inherited. This is kind of confusing -- only the attribute default would be overridden.

Here's a suggested simpler example:

class Window:
    BORDER_WIDTH: Final = 2
    ...

class ListView(Window):
    BORDER_WIDTH = 3  # Error: can't override a final attribute
    ...

"""An absolutely unique object in the database"""
def __init__(self) -> None:
self.id: Final = uuid.uuid4()

# 1000 lines later

class User(Snowflake):
id = uuid.uuid4() # Error: can't override a final attribute

Some other use cases might be solved by using ``@property``, but note that
neither of the above use cases can be solved with it.

.. note::

This is an experimental feature. Some details might change in later
versions of mypy. The final qualifiers are available in ``typing_extensions``
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
module.

Definition syntax
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
*****************

The ``typing_extensions.Final`` qualifier indicates that a given name or
attribute should never be re-assigned, re-defined, nor overridden. It can be
used in one of these forms:


* You can provide an explicit type using the syntax ``Final[<type>]``. Example:

.. code-block:: python

ID: Final[float] = 1

* You can omit the type: ``ID: Final = 1``. Note that unlike for generic
classes this is *not* the same as ``Final[Any]``. Here mypy will infer
type ``int``.

* In stub files one can omit the right hand side and just write
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
``ID: Final[float]``.

* Finally, one can define ``self.id: Final = 1`` (also with a type argument),
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
but this is allowed *only* in ``__init__`` methods.

Definition rules
****************

The are two rules that should be always followed when defining a final name:

* There can be *at most one* final declaration per module or class for
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd leave this section out as too much detail. The same restriction applies to other things such as classes and I don't think that we mention it anywhere. It should be obvious why mypy rejects two final definitions, and the error message is probably clear enough.

a given attribute:

.. code-block:: python

from typing_extensions import Final

ID: Final = 1
ID: Final = 2 # Error: "ID" already declared as final

class SomeCls:
id: Final = 1
def __init__(self, x: int) -> None:
self.id: Final = x # Error: "id" already declared in class body

Note that mypy has a single namespace for a class. So there can't be two
class-level and instance-level constants with the same name.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be a bit clearer to write this as "... there can't be a class-level and an instance-level constant with the same name". This could also be left out -- it's a bit odd that we talk about the single namespace in detail only here, whereas we probably don't mention it when talking about how to define classes in general.


* There must be *exactly one* assignment to a final attribute:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, maybe this is too much detail. If a user accidentally assigns twice, the error message from mypy should be sufficient. If not, we should improve the error instead of explaining it in the documentation.


.. code-block:: python

ID = 1
ID: Final = 2 # Error!

class SomeCls:
ID = 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use a different name here to make it obvious that this is a different attribute? Somebody reading this quickly might not notice that there are two attributes with the same name in different scopes.

ID: Final = 2 # Error!

* A final attribute declared in class body without r.h.s. must be initialized
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
in the ``__init__`` method (one can skip initializer in stub files):
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: python

class SomeCls:
x: Final
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
y: Final # Error: final attribute without an initializer
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
def __init__(self) -> None:
self.x = 1 # Good

* ``Final`` can be only used as an outermost type in assignments, using it in
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grammar nit: run on sentence -- write as "... assignments. Using ...".

Somebody may think that this conflicts with x: Final being valid. Maybe write as "... in assignments or variable annotations".

any other position is an error. In particular, ``Final`` can't be used in
annotations for function arguments because this may cause confusions about
what are the guarantees in this case:
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: python

x: List[Final[int]] = [] # Error!
def fun(x: Final[List[int]]) -> None: # Error!
...

* ``Final`` and ``ClassVar`` should not be used together. Mypy will infer
the scope of a final declaration automatically depending on whether it was
initialized in class body or in ``__init__``.
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved

.. note::
Conditional final declarations and final declarations within loops are
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another thing that we could perhaps leave out since other parts of the documentation don't go into this much detail. In general, I don't like spelling out many limitations in detail, since it makes it sound like mypy is doing a poor job, even though most of these limitations don't affect most users or have a good reason to exist. When a user tries to do something that's not supported, mypy will complain anyway.

rejected.

Using final attributes
**********************

As a result of a final declaration mypy strives to provide the
two following guarantees:

* A final attribute can't be re-assigned (or otherwise re-defined), both
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On further reflection, maybe this is too much detail, as we already mentioned most of this before. I think that this item and the example can be left out.

internally and externally:

.. code-block:: python

# file mod.py
from typing_extensions import Final

ID: Final = 1

class SomeCls:
ID: Final = 1

def meth(self) -> None:
self.ID = 2 # Error: can't assign to final attribute

# file main.py
import mod
mod.ID = 2 # Error: can't assign to constant.

from mod import ID
ID = 2 # Also an error, see note below.

class DerivedCls(mod.SomeCls):
...

DerivedCls.ID = 2 # Error!
obj: DerivedCls
obj.ID = 2 # Error!

* A final attribute can't be overridden by a subclass (even with another
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
explicit final declaration). Note however, that final attributes can
override read-only properties. This also applies to multiple inheritance:

.. code-block:: python

class Base:
@property
def ID(self) -> int: ...

class One(Base):
ID: Final = 1 # OK

class Other(Base):
ID: Final = 2 # OK

class Combo(One, Other): # Error: cannot override final attribute.
pass

* Declaring a name as final only guarantees that the name wll not be re-bound
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
to other value, it doesn't make the value immutable. One can use immutable ABCs
and containers to prevent mutating such values:

.. code-block:: python

x: Final = ['a', 'b']
x.append('c') # OK

y: Final[Sequance[str]] = ['a', 'b']
y.append('x') # Error: Sequance is immutable
z: Final = ('a', 'b') # Also an option

.. note::

Mypy treats re-exported final names as final. In other words, once declared,
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
the final status can't be "stripped". Such behaviour is typically desired
for larger libraries where constants are defined in a separate module and
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
then re-exported.

Final methods
*************

Like with attributes, sometimes it is useful to protect a method from
overriding. In such situations one can use a ``typing_extensions.final``
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
decorator:

.. code-block:: python

from typing_extensions import final

class Base:
@final
def common_name(self) -> None:
...

# 1000 lines later

class Derived(Base):
def common_name(self) -> None: # Error: this overriding might break
# invariants in the base class.
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
...

This ``@final`` decorator can be used with instance methods, class methods,
static methods, and properties (this includes overloaded methods). For
overloaded methods one should add ``@final`` on the implementation to make
it final (or on the first overload in stubs):

.. code-block:: python
from typing import Any, overload

class Base:
@overload
def meth(self) -> None: ...
@overload
def meth(self, arg: int) -> int: ...
@final
def meth(self, x=None):
...

class Derived(Base):
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
def meth(self, x: Any = None) -> Any: # Error: can't override final method
...

Final classes
*************

You can apply a ``typing_extensions.final`` decorator to a class indicates
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved
to mypy that it can't be subclassed. The decorator acts as a declaration
for mypy (and as documentation for humans), but it doesn't prevent subclassing
at runtime:

.. code-block:: python

from typing_extensions import final

@final
class Leaf:
...

from lib import Leaf
ilevkivskyi marked this conversation as resolved.
Show resolved Hide resolved

class MyLeaf(Leaf): # Error: Leaf can't be subclassed
...

Here are some situations where using a final class may be useful:

* A class wasn't designed to be subclassed. Perhaps subclassing does not
work as expected, or it's error-prone.
* You want to retain the freedom to arbitrarily change the class implementation
in the future, and these changes might break subclasses.
* You believe that subclassing would make code harder to understand or maintain.
For example, you may want to prevent unnecessarily tight coupling between
base classes and subclasses.
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Mypy is a static type checker for Python 3 and Python 2.7.
stubs
generics
more_types
final_attrs
metaclasses

.. toctree::
Expand Down
Loading