Typing Cleanup - Remove Blacklisted Items

[WillAyd]

#25844 introduced a blacklist of modules in the mypy.ini config file which currently throw errors when analyzed with mypy. We would ideally be able to whittle these down (save exclusion of the test folder(s), which I think is fine).

I'm not sure there is one easy way to split it up as the degree of difficulty may vary wildly. PRs from the community to clean up one to a few of the blacklisted modules at a time would certainly be welcome!

Here's the list of modules for reference:

• pandas._config.config (Fix Type Annotations in pandas/_config/config.py #25899)
• pandas._version (Fix type annotation in pandas. version #25949)
• pandas.compat (Fix Type Annotation in pandas.compat #26251)
• pandas.compat.numpy.function
• pandas.compat.pickle_compat (Type annotation pandas/compat/pickle compat.py #25991)
• pandas.core.accessor (Fix Type Annotation in pandas/core/accessor.py #26146)
• pandas.core.api
• pandas.core.apply
• pandas.core.arrays.array_ (Fix Type Annotations in pandas.core.arrays #26065)
• pandas.core.arrays.datetimelike (Fix Type Annotations in pandas.core.arrays #26065)
• pandas.core.arrays.integer (Fix Type Annotations in pandas.core.arrays #26065)
• pandas.core.arrays.interval (Fix Type Annotations in pandas.core.arrays #26065)
• pandas.core.arrays.numpy_ (Fix Type Annotations in pandas.core.arrays #26065)
• pandas.core.arrays.period (Fix Type Annotations in pandas.core.arrays #26065)
• pandas.core.arrays.sparse (Fix Type Annotations in pandas.core.arrays #26065)
• pandas.core.arrays.timedeltas (Fix Type Annotations in pandas.core.arrays #26065)
• pandas.core.base
• pandas.core.computation.expr
• pandas.core.computation.ops
• pandas.core.computation.pytables
• pandas.core.config (Removed resolved items from blacklist #26048)
• pandas.core.config_init
• pandas.core.dtypes.dtypes (Fix Type Annotations in pandas.core.dtypes #26028)
• pandas.core.dtypes.missing (Fix Type Annotations in pandas.core.dtypes #26028)
• pandas.core.frame
• pandas.core.generic
• pandas.core.groupby.generic (Fix Up Typing in GroupBy #26089)
• pandas.core.groupby.groupby (Fix Up Typing in GroupBy #26089)
• pandas.core.groupby.ops (Fix Up Typing in GroupBy #26089)
• pandas.core.indexes.base (Remove other pandas.core.index files from MyPy Blacklist  #26315)
• pandas.core.indexes.datetimelike
• pandas.core.indexes.datetimes (Remove other pandas.core.index files from MyPy Blacklist  #26315)
• pandas.core.indexes.period (Remove other pandas.core.index files from MyPy Blacklist  #26315)
• pandas.core.indexes.timedeltas (Remove other pandas.core.index files from MyPy Blacklist  #26315)
• pandas.core.indexing
• pandas.core.internals.blocks
• pandas.core.ops
• pandas.core.panel (DEPRECATED - do not update)
• pandas.core.resample
• pandas.core.reshape.concat (TYP: Replace wildcardimports in toplevelinit as precursor for reshape,stata,io PRs #25936 #25940 #25939 #26019)
• pandas.core.reshape.merge
• pandas.core.reshape.reshape
• pandas.core.reshape.tile (TYP: Replace wildcardimports in toplevelinit as precursor for reshape,stata,io PRs #25936 #25940 #25939 #26019)
• pandas.core.series
• pandas.core.sparse.frame
• pandas.core.strings (Fix Type Annotations in pandas.core.strings #25902)
• pandas.core.util.hashing
• pandas.core.window
• pandas.io.clipboards (Removed resolved items from blacklist #26048)
• pandas.io.feather_format (TYP: Assorted_Typing_Fix in pandas.io #25939)
• pandas.io.formats.css (Remove pandas.io.formats.css from MyPy Blacklist #25900)
• pandas.io.html (TYP: Assorted_Typing_Fix in pandas.io #25939)
• pandas.io.json.json (TYP: Assorted_Typing_Fix in pandas.io #25939)
• pandas.io.json.normalize (TYP: Assorted_Typing_Fix in pandas.io #25939)
• pandas.io.json.table_schema (TYP: Assorted_Typing_Fix in pandas.io #25939)
• pandas.io.packers (TYP: Assorted_Typing_Fix in pandas.io #25939)
• pandas.io.parquet (TYP: Assorted_Typing_Fix in pandas.io #25939)
• pandas.io.pytables
• pandas.io.stata (TYP: Remove pandas.io.stata from Typing Blacklist #25940)
• pandas.plotting._core (#26020 Fixed type annotations for pandas.plotting._core #26021)
• pandas.tseries.frequencies (GH26013 Fix type annotations in pandas/tseries/* #26014)
• pandas.tseries.holiday (GH26013 Fix type annotations in pandas/tseries/* #26014)
• pandas.tseries.offsets (GH26013 Fix type annotations in pandas/tseries/* #26014)
• pandas.util._doctools
• pandas.util.testing

[GreybeardCG]

@WillAyd I've been studying a few of these files and related contributions today and was wondering if there are any catches I need to know about when fixing these mypy violations.
As an example: in array_.py:231 adding dtype = cast(ExtensionDtype, dtype)" seems the least intrusive. However, I saw that typing was added to this file partially already , so was there a specific reason to leave this violation in? Thanks a lot!

[WillAyd]

Have you merged master?

[GreybeardCG]

@WillAyd Yes, I'm on the latest upstream/master.

[WillAyd]

Sorry misread comment. We’ve had a few things on one of which was simply to compat syntax to Py3 annotation

For the module you mentioned we did that but haven’t focused on actually fixing the errors. If you still see some and want to submit a PR would certainly be welcome

[GreybeardCG]

@WillAyd All clear then! I'll keep working on it.

[gwrome]

@WillAyd I've run into a few cases where we should use an old style type hint for a global variable to maintain pre-3.6 compatibility, but the appropriate hint is just too long for the line. I haven't been able to find any documented way to break a variable's type hint across lines or put it on another line.

Any ideas?

For example in pandas/compat/numpy/function.py:105 we want something like

ARGSORT_DEFAULTS = OrderedDict()  # type: OrderedDict[str, Union[int, str, None]]

but that line is too long. We could replace the Union with Any, but that reduces the usefulness of type checking. In this particular case, it probably doesn't matter because that ARGSORT_DEFAULTS gets fed directly into a classconstructor and is never touched again. But there may be other places in the code where accuracy is more important and the line length is an issue.

[Naddiseo]

@gwrome python/mypy#4511 might be of use

tldr: type alias, or backslash at end of line