-
-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make --namespace-packages the default #9636
Conversation
Great stuff! Some of the test failures can be tricky to debug; I can help with them. |
Yeah, I'd love help here, maybe you could take a look at testAddedMissingStubsPackage or testDeletePackage5? |
Sure, I should have time to look at these later this week. |
9d6b6dc
to
5275932
Compare
Spent some time looking at this today. I was able to fix the "cache data discrepancy" errors. I also spent some time looking at mypy daemon / fine grained errors, but that turned out to be a real rabbit hole. Since |
This comment has been minimized.
This comment has been minimized.
Unfortunately, a number of tests fail. Please help fixing them :-) Especially the ones to do with cache inconsistencies... There's at least one known bug here: python#7276
This should cover the current state on master, as implemented across python#9742, python#9683, python#9632, python#9616, python#9614, etc. This will need to be changed if we can make `--namespace-packages` the default (python#9636).
This should cover the current state on master, as previously discussed / implemented across python#9742, python#9683, python#9632, python#9616, python#9614, etc. This will need to be changed if we can make `--namespace-packages` the default (python#9636). I haven't documented some of the finer points of the changes, since it felt like an inappropriate level of detail (e.g. using absolute paths when crawling, how directories with invalid package names affect crawling, etc)
This should cover the current state on master, as previously discussed / implemented across python#9742, python#9683, python#9632, python#9616, python#9614, etc. This will need to be changed if we can make `--namespace-packages` the default (python#9636). I haven't documented some of the finer points of the changes, since it felt like an inappropriate level of detail (e.g. using absolute paths when crawling, how directories with invalid package names affect crawling, etc)
This should cover the current state on master, as previously discussed / implemented across #9742, #9683, #9632, #9616, #9614, etc. This will need to be changed if we can make `--namespace-packages` the default (#9636). I haven't documented some of the finer points of the changes, since it felt like an inappropriate level of detail (e.g. using absolute paths when crawling, how directories with invalid package names affect crawling, etc) Co-authored-by: hauntsaninja <>
This should cover the current state on master, as previously discussed / implemented across #9742, #9683, #9632, #9616, #9614, etc. This will need to be changed if we can make `--namespace-packages` the default (#9636). I haven't documented some of the finer points of the changes, since it felt like an inappropriate level of detail (e.g. using absolute paths when crawling, how directories with invalid package names affect crawling, etc) Co-authored-by: hauntsaninja <>
Sorry, I lost of track of this PR. Is there anything I can do to help move this forward? |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
…re needed This matters with namespace packages - since an empty package with some .2 files in it will be considered a namespace package. This is truer - by only writing the files at the point of the step. This helps unblock python#9636
@@ -516,7 +516,7 @@ def add_invertible_flag(flag: str, | |||
title='Import discovery', | |||
description="Configure how imports are discovered and followed.") | |||
add_invertible_flag( | |||
'--namespace-packages', default=False, | |||
'--no-namespace-packages', dest="namespace_packages", default=True, | |||
help="Support namespace packages (PEP 420, __init__.py-less)", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, updated!
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Might help with breaking changes that result from python#9636
Might help with breaking changes that result from #9636 Co-authored-by: hauntsaninja <>
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Can this be merged? |
Wanted to check in here to see if its possible to go forward with this. |
Diff from mypy_primer, showing the effect of this PR on open source code: pandera (https://github.com/pandera-dev/pandera)
+ docs/source/conf.py:108: error: Need type annotation for "exclude_patterns" (hint: "exclude_patterns: List[<type>] = ...") [var-annotated]
rotki (https://github.com/rotki/rotki)
+ tools/data_faker/data_faker/mock_apis/encoding.py:8: error: Function is missing a type annotation
+ tools/data_faker/data_faker/mock_apis/encoding.py:17: error: Function is missing a type annotation
+ tools/data_faker/data_faker/mock_apis/resources.py:16: error: Function is missing a type annotation
+ tools/data_faker/data_faker/mock_apis/resources.py:22: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/resources.py:27: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/resources.py:32: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/resources.py:37: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/resources.py:43: error: Function is missing a type annotation
+ tools/data_faker/data_faker/mock_apis/resources.py:62: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/resources.py:67: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/resources.py:74: error: Function is missing a type annotation
+ tools/data_faker/data_faker/mock_apis/api.py:43: error: Function is missing a type annotation
+ tools/data_faker/data_faker/mock_apis/api.py:53: error: Function is missing a type annotation
+ tools/data_faker/data_faker/mock_apis/api.py:54: error: Call to untyped function "api_error" in typed context
+ tools/data_faker/data_faker/mock_apis/api.py:57: error: Function is missing a type annotation for one or more arguments
+ tools/data_faker/data_faker/mock_apis/api.py:68: error: Function is missing a type annotation for one or more arguments
+ tools/data_faker/data_faker/mock_apis/api.py:87: error: Function is missing a type annotation for one or more arguments
+ tools/data_faker/data_faker/mock_apis/api.py:107: error: Function is missing a type annotation for one or more arguments
+ tools/data_faker/data_faker/mock_apis/api.py:111: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/api.py:114: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/api.py:117: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/api.py:120: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/api.py:123: error: Function is missing a type annotation
+ tools/data_faker/data_faker/mock_apis/api.py:126: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/api.py:129: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/mock_apis/api.py:132: error: Function is missing a type annotation
+ tools/data_faker/data_faker/statistics.py:19: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/fake_binance.py:117: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/fake_binance.py:147: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/fake_binance.py:150: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/fake_binance.py:168: error: Incompatible types in assignment (expression has type "List[Dict[str, str]]", target has type "int")
+ tools/data_faker/data_faker/fake_binance.py:171: error: Function is missing a type annotation
+ tools/data_faker/data_faker/actions.py:33: error: Function is missing a type annotation for one or more arguments
+ tools/data_faker/data_faker/actions.py:122: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/actions.py:163: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/fake_kraken.py:105: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/fake_kraken.py:119: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/fake_kraken.py:122: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/fake_kraken.py:125: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/fake_kraken.py:129: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/fake_kraken.py:134: error: Function is missing a return type annotation
+ tools/data_faker/data_faker/faker.py:68: error: Function is missing a return type annotation
kornia (https://github.com/kornia/kornia)
+ kornia/utils/_compat.py:6: error: Module "packaging" has no attribute "version" [attr-defined]
rclip (https://github.com/yurijmikhalevich/rclip)
+ rclip/db.py: error: Source file found twice under different module names: "db" and "rclip.db"
- rclip/model.py:3: error: Cannot find implementation or library stub for module named "clip"
- rclip/model.py:4: error: Cannot find implementation or library stub for module named "clip.model"
- rclip/model.py:5: error: Cannot find implementation or library stub for module named "numpy"
- rclip/model.py:6: error: Cannot find implementation or library stub for module named "PIL"
- rclip/model.py:7: error: Cannot find implementation or library stub for module named "rclip"
- rclip/model.py:8: error: Cannot find implementation or library stub for module named "torch"
- rclip/model.py:9: error: Cannot find implementation or library stub for module named "torch.nn"
- rclip/db.py:70: error: Argument 1 to "update" of "MutableMapping" has incompatible type "NewImage"; expected "SupportsKeysAndGetItem[str, None]"
- rclip/db.py:70: note: Following member(s) of "NewImage" have conflicts:
- rclip/db.py:70: note: Expected:
- rclip/db.py:70: note: def __getitem__(self, str) -> None
- rclip/db.py:70: note: Got:
- rclip/db.py:70: note: def __getitem__(self, str) -> object
- rclip/utils.py:4: error: Cannot find implementation or library stub for module named "PIL"
- rclip/utils.py:6: error: Library stubs not installed for "requests" (or incompatible with Python 3.10)
- rclip/utils.py:6: note: Hint: "python3 -m pip install types-requests"
- rclip/utils.py:6: note: (or run "mypy --install-types" to install all missing stub packages)
- rclip/utils.py:9: error: Cannot find implementation or library stub for module named "rclip"
- rclip/utils.py:42: error: Incompatible types in assignment (expression has type "Path", variable has type "Optional[str]")
- rclip/utils.py:45: error: Argument 1 to "makedirs" has incompatible type "Optional[str]"; expected "Union[str, bytes, PathLike[str], PathLike[bytes]]"
- rclip/utils.py:46: error: Incompatible return value type (got "Optional[str]", expected "Path")
- rclip/main.py:7: error: Cannot find implementation or library stub for module named "numpy"
- rclip/main.py:7: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
+ rclip/db.py: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-file-paths-to-modules for more info
+ rclip/db.py: note: Common resolutions include: a) adding `__init__.py` somewhere, b) using `--explicit-package-bases` or adjusting MYPYPATH
- rclip/main.py:8: error: Cannot find implementation or library stub for module named "tqdm"
- rclip/main.py:9: error: Cannot find implementation or library stub for module named "PIL"
- rclip/main.py:12: error: Cannot find implementation or library stub for module named "rclip"
- rclip/main.py:32: error: TypedDict key must be a string literal; expected one of ("modified_at", "size")
jax (https://github.com/google/jax)
- jax/_src/lib/mlir/dialects/__init__.py:16: error: Cannot find implementation or library stub for module named "jaxlib" [import]
- jax/_src/lib/mlir/__init__.py:16: error: Cannot find implementation or library stub for module named "jaxlib" [import]
+ jaxlib/pocketfft.py:37: error: Variable "jaxlib.pocketfft.FftType" is not valid as a type [valid-type]
+ jaxlib/pocketfft.py:37: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
+ jaxlib/pocketfft.py:77: error: Incompatible return value type (got "Tuple[bytes, Any, List[int]]", expected "bytes") [return-value]
+ jaxlib/pocketfft.py:107: error: Incompatible return value type (got "Tuple[Any, Any, List[int]]", expected "bytes") [return-value]
+ jaxlib/pocketfft.py:110: error: Variable "jaxlib.pocketfft.FftType" is not valid as a type [valid-type]
+ jaxlib/pocketfft.py:110: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
+ jaxlib/pocketfft.py:130: error: Unsupported right operand type for in ("int") [operator]
- jax/experimental/jax2tf/converters_eval/models_test.py:36: error: Cannot find implementation or library stub for module named "jax.experimental.jax2tf.converters_eval" [import]
- jax/experimental/jax2tf/converters_eval/models_test.py:42: error: Cannot find implementation or library stub for module named "jax.experimental.jax2tf.converters_eval.models" [import]
- jax/_src/lib/__init__.py:38: error: Cannot find implementation or library stub for module named "jaxlib" [import]
- jax/_src/lib/__init__.py:101: error: Cannot find implementation or library stub for module named "jaxlib.lapack" [import]
- jax/_src/lib/__init__.py:102: error: Cannot find implementation or library stub for module named "jaxlib.pocketfft" [import]
- jax/_src/lib/__init__.py:109: error: Cannot find implementation or library stub for module named "jaxlib.gpu_solver" [import]
- jax/_src/lib/__init__.py:110: error: Cannot find implementation or library stub for module named "jaxlib.gpu_sparse" [import]
- jax/_src/lib/__init__.py:111: error: Cannot find implementation or library stub for module named "jaxlib.gpu_prng" [import]
- jax/_src/lib/__init__.py:112: error: Cannot find implementation or library stub for module named "jaxlib.gpu_linalg" [import]
- jax/experimental/jax2tf/converters_eval/models.py:22: error: Cannot find implementation or library stub for module named "jax.experimental.jax2tf.converters_eval.test_models.flax" [import]
- jax/experimental/jax2tf/converters_eval/converters.py:23: error: Cannot find implementation or library stub for module named "jax.experimental.jax2tf.converters_eval.models" [import]
+ jax/experimental/jax2tf/examples/tf_js/quickdraw/quickdraw.py:38: error: Cannot find implementation or library stub for module named "utils" [import]
streamlit (https://github.com/streamlit/streamlit)
- lib/tests/streamlit/watcher/test_data/nested_module_parent.py:15:1: error: Cannot find implementation or library stub for module named "tests.streamlit.watcher.test_data.nested_module_child" [import]
- lib/tests/streamlit/watcher/test_data/nested_module_parent.py:15:1: error: Cannot find implementation or library stub for module named "tests.streamlit.watcher.test_data" [import]
- lib/tests/streamlit/watcher/test_data/nested_module_parent.py:15:1: error: Cannot find implementation or library stub for module named "tests.streamlit.watcher" [import]
- lib/tests/streamlit/watcher/test_data/nested_module_parent.py:15:1: error: Cannot find implementation or library stub for module named "tests.streamlit" [import]
- lib/tests/streamlit/watcher/local_sources_watcher_test.py:27:1: error: Cannot find implementation or library stub for module named "tests.streamlit.watcher.test_data.dummy_module1" [import]
- lib/tests/streamlit/watcher/local_sources_watcher_test.py:27:1: error: Cannot find implementation or library stub for module named "tests.streamlit.watcher.test_data" [import]
- lib/tests/streamlit/watcher/local_sources_watcher_test.py:27:1: error: Cannot find implementation or library stub for module named "tests.streamlit.watcher" [import]
- lib/tests/streamlit/watcher/local_sources_watcher_test.py:27:1: error: Cannot find implementation or library stub for module named "tests.streamlit" [import]
- lib/tests/streamlit/watcher/local_sources_watcher_test.py:28:1: error: Cannot find implementation or library stub for module named "tests.streamlit.watcher.test_data.dummy_module2" [import]
- lib/tests/streamlit/watcher/local_sources_watcher_test.py:29:1: error: Cannot find implementation or library stub for module named "tests.streamlit.watcher.test_data.misbehaved_module" [import]
- lib/tests/streamlit/watcher/local_sources_watcher_test.py:30:1: error: Cannot find implementation or library stub for module named "tests.streamlit.watcher.test_data.nested_module_parent" [import]
- lib/tests/streamlit/watcher/local_sources_watcher_test.py:31:1: error: Cannot find implementation or library stub for module named "tests.streamlit.watcher.test_data.nested_module_child" [import]
- lib/tests/streamlit/watcher/local_sources_watcher_test.py: note: At top level:
- lib/tests/streamlit/watcher/local_sources_watcher_test.py:274:1: error: Cannot find implementation or library stub for module named "tests.streamlit.watcher.test_data.namespace_package" [import]
+ lib/tests/streamlit/watcher/local_sources_watcher_test.py: note: In member "test_namespace_package_unloaded" of class "LocalSourcesWatcherTest":
+ lib/tests/streamlit/watcher/local_sources_watcher_test.py:276:36: error: "MutableSequence[str]" has no attribute "_path" [attr-defined]
- lib/tests/streamlit/echo_test.py:79:1: error: Cannot find implementation or library stub for module named "tests.streamlit.echo_test_data.root_level_echo" [import]
- lib/tests/streamlit/echo_test.py:79:1: error: Cannot find implementation or library stub for module named "tests.streamlit.echo_test_data" [import]
- lib/tests/streamlit/echo_test.py:79:1: error: Cannot find implementation or library stub for module named "tests.streamlit" [import]
+ lib/tests/streamlit/components_test.py:70:1: error: Cannot find implementation or library stub for module named "component_test_data" [import]
+ lib/tests/streamlit/components_test.py:78:1: error: Cannot find implementation or library stub for module named "component_test_data.outer_module" [import]
+ lib/tests/streamlit/components_test.py:86:1: error: Cannot find implementation or library stub for module named "component_test_data.nested.inner_module" [import]
- lib/tests/streamlit/state/session_state_test.py:41:1: error: Cannot find implementation or library stub for module named "tests.streamlit.state.strategies" [import]
- lib/tests/streamlit/state/session_state_test.py:41:1: error: Cannot find implementation or library stub for module named "tests.streamlit.state" [import]
- lib/tests/streamlit/state/session_state_test.py:41:1: error: Cannot find implementation or library stub for module named "tests.streamlit" [import]
manticore (https://github.com/trailofbits/manticore)
- tests/auto_generators/make_dump.py:360: error: "object" has no attribute "__iter__"; maybe "__str__" or "__dir__"? (not iterable)
- tests/auto_generators/make_dump.py:362: error: "object" has no attribute "__iter__"; maybe "__str__" or "__dir__"? (not iterable)
- tests/auto_generators/make_dump.py:365: error: "object" has no attribute "__iter__"; maybe "__str__" or "__dir__"? (not iterable)
- tests/auto_generators/make_dump.py:416: error: "object" has no attribute "__iter__"; maybe "__str__" or "__dir__"? (not iterable)
- tests/auto_generators/make_dump.py:418: error: "object" has no attribute "__iter__"; maybe "__str__" or "__dir__"? (not iterable)
- tests/auto_generators/make_dump.py:428: error: "object" has no attribute "__iter__"; maybe "__str__" or "__dir__"? (not iterable)
discord.py (https://github.com/Rapptz/discord.py)
- discord/app_commands/commands.py:71: error: Cannot find implementation or library stub for module named "discord.ext.commands"
- discord/app_commands/commands.py:71: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
- discord/ext/commands/parameters.py:270: error: Incompatible types in assignment (expression has type "OrderedDict[str, commands.parameters.Parameter]", base class "Signature" defined the type as "MappingProxyType[str, inspect.Parameter]")
+ discord/ext/commands/parameters.py:270: error: Incompatible types in assignment (expression has type "OrderedDict[str, discord.ext.commands.parameters.Parameter]", base class "Signature" defined the type as "MappingProxyType[str, inspect.Parameter]")
- discord/ext/commands/cog.py:338: error: Incompatible types in assignment (expression has type "Union[Group, discord.app_commands.commands.Command[Self?, [VarArg(Any), KwArg(Any)], Any]]", variable has type "commands.core.Command[Self?, [VarArg(Any), KwArg(Any)], Any]")
+ discord/ext/commands/cog.py:338: error: Incompatible types in assignment (expression has type "Union[Group, discord.app_commands.commands.Command[Self?, [VarArg(Any), KwArg(Any)], Any]]", variable has type "discord.ext.commands.core.Command[Self?, [VarArg(Any), KwArg(Any)], Any]")
- discord/ext/commands/cog.py:645: error: Incompatible types in assignment (expression has type "Union[Group, discord.app_commands.commands.Command[Self?, [VarArg(Any), KwArg(Any)], Any]]", variable has type "commands.core.Command[Self?, [VarArg(Any), KwArg(Any)], Any]")
+ discord/ext/commands/cog.py:645: error: Incompatible types in assignment (expression has type "Union[Group, discord.app_commands.commands.Command[Self?, [VarArg(Any), KwArg(Any)], Any]]", variable has type "discord.ext.commands.core.Command[Self?, [VarArg(Any), KwArg(Any)], Any]")
- discord/ext/commands/cog.py:647: error: Argument 1 to "add_command" of "CommandTree" has incompatible type "commands.core.Command[Self?, [VarArg(Any), KwArg(Any)], Any]"; expected "Union[discord.app_commands.commands.Command[Any, [VarArg(Any), KwArg(Any)], Any], ContextMenu, Group]"
+ discord/ext/commands/cog.py:647: error: Argument 1 to "add_command" of "CommandTree" has incompatible type "discord.ext.commands.core.Command[Self?, [VarArg(Any), KwArg(Any)], Any]"; expected "Union[discord.app_commands.commands.Command[Any, [VarArg(Any), KwArg(Any)], Any], ContextMenu, Group]"
- discord/ext/commands/cog.py:660: error: Incompatible types in assignment (expression has type "Union[Group, discord.app_commands.commands.Command[Self?, [VarArg(Any), KwArg(Any)], Any]]", variable has type "commands.core.Command[Self?, [VarArg(Any), KwArg(Any)], Any]")
+ discord/ext/commands/cog.py:660: error: Incompatible types in assignment (expression has type "Union[Group, discord.app_commands.commands.Command[Self?, [VarArg(Any), KwArg(Any)], Any]]", variable has type "discord.ext.commands.core.Command[Self?, [VarArg(Any), KwArg(Any)], Any]")
- discord/ext/commands/cooldowns.py:34: error: Skipping analyzing ".abc": module is installed, but missing library stubs or py.typed marker
- discord/ext/commands/cooldowns.py:34: error: Relative import climbs too many namespaces
- discord/ext/commands/cooldowns.py:42: error: Skipping analyzing ".message": module is installed, but missing library stubs or py.typed marker
- discord/ext/commands/cooldowns.py:42: error: Relative import climbs too many namespaces
+ discord/ext/commands/core.py:413: error: Incompatible types in assignment (expression has type "CooldownMapping[Union[Message, Context[BotT]]]", variable has type "CooldownMapping[Context[Any]]")
freqtrade (https://github.com/freqtrade/freqtrade)
+ freqtrade/__init__.py: error: Source file found twice under different module names: "freqtrade" and "freqtrade.__init__"
+ freqtrade/__init__.py: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-file-paths-to-modules for more info
+ freqtrade/__init__.py: note: Common resolutions include: a) adding `__init__.py` somewhere, b) using `--explicit-package-bases` or adjusting MYPYPATH
|
Looks like this caused the selfcheck to start failing on master: https://github.com/python/mypy/actions/runs/3133286021/jobs/5086495727 |
Fixed in #13748 |
Unfortunately, a number of tests fail. Please help fixing them :-)
Especially the ones to do with cache inconsistencies... There's at least
one known bug here: #7276