Repository navigation
Python 3.5 failing with AttributeError: module 'os' has no attribute 'PathLike' #1219
Description
Activity
Thanks for reporting.
That's interesting as I would have thought that our CI can pick this up.
Can it be that PathLike was backported and a more recent version of python 3.5 supports it?In any case, since python 3.5 is EOL and this PR exists I would probably prefer to cancel python 3.5 support rather than undoing the use of PathLike.
In order to get this fixed, I could yank the offending version. However, first I would love to know why our CI seems to be fine with it.
Thanks for your help
We are using default in Azure Pipelines for python3, that seems to be 3.5 for us.
We just patched to use python 3.6, so we are no longer impacted by 3.5.
Good to hear.
Since there is no further information on how to reproduce this issue, as well as no error messages for others to find, I think it's ok to close this issue until further information is presented to help understanding what's going on.
Thanks for your understanding.
FYI: I think this issue needs to be re-opened. It's still a problem with Python 3.6 too, just a different exception. (See below for details).
That's interesting as I would have thought that our CI can pick this up.
Can it be that PathLike was backported and a more recent version of python 3.5 supports it?Doesn't appear to be the case:
$ docker run -it --rm python:3.5-slim Python 3.5.10 (default, Sep 10 2020, 18:40:42) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from os import PathLike Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: cannot import name 'PathLike' >>>In any case, since python 3.5 is EOL and this PR exists I would probably prefer to cancel python 3.5 support rather than undoing the use of PathLike.
This would seem like something unexpected to do in a patch release. Also, I find this to be broken for more than 3.5...
In order to get this fixed, I could yank the offending version. However, first I would love to know why our CI seems to be fine with it.
I think it's because of this in git/types.py:
PathLike = Union[str, 'os.PathLike[str]']
For some reason that was written as a forward reference instead of a direct reference. I don't see any need for that here. This would cause evaluation of the type expression to be delayed until it's evaluated:
$ docker run -it --rm python:3.5-slim Python 3.5.10 (default, Sep 10 2020, 18:40:42) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from typing import Union >>> import os >>> PathLike = Union[str, os.PathLike[str]] Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: module 'os' has no attribute 'PathLike' >>> PathLike = Union[str, 'os.PathLike[str]'] >>> PathLike.__args__[1]._eval_type(globals(), locals()) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.5/typing.py", line 245, in _eval_type eval(self.__forward_code__, globalns, localns), File "<string>", line 1, in <module> AttributeError: module 'os' has no attribute 'PathLike' >>>As you can see this fails immediately without using a forward reference (string version of the type expression) but waits to fail until evaluated by
typing.pywhen using a forward reference.With Python 3.6 we get some improvement, as it's no longer complaining about
PathLikenot existing. Instead it complains about that not being subscriptable... (I'm skipping the forward reference here for brevity, but it's the same when still using that, it just hides the problem):$ docker run -it --rm python:3.6-slim Python 3.6.12 (default, Dec 11 2020, 15:17:39) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from typing import Union >>> import os >>> PathLike = Union[str, os.PathLike[str]] Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'ABCMeta' object is not subscriptableThis exact error continues up to and including 3.8, it's only starting from 3.9 that both forms succeed:
$ docker run -it --rm python:3.9-slim Python 3.9.1 (default, Jan 12 2021, 16:56:42) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from typing import Union >>> import os >>> PathLike = Union[str, os.PathLike[str]] >>> PathLike = Union[str, 'os.PathLike[str]'] >>> PathLike.__args__[1]._evaluate(globals(), locals(), frozenset()) os.PathLike[str] >>>The reason why none of this fails is because forward references will only get evaluated by tools that actually use the type annotations. And while I see a config file for mypy, which is such a tool, I cannot find any place where mypy actually gets called from CI...
Summary:
- for 3.5 it's the non-existence of
os.PathLikethat's a problem. - for 3.6, 3.7 and 3.8 it's the fact that it's not subscriptable (with
str) asos.PathLike[str].
Unfortunately I cannot find any documentation about the subscript form for that type, so I wouldn't know directly how to fix it for 3.6 onwards, let alone 3.5...
I'm not even sure exactly what the intent of @Yobmod is with using the subscript form... I suggest poking him to look at this and fix it, and if that takes too long replacing the
'os.PathLike[str]'expression withTBD(with a comment) in the interim.- for 3.5 it's the non-existence of
In
git/compat.py
from git.types import TBDin
git/types.pyfrom typing import Union, Any TBD = Any PathLike = Union[str, 'os.PathLike[str]']os.PathLike does not exist in python 3.5..
So importing types should create this error in 3.5.
Thanks a lot, @c4xuxo and @muggenhor. I think this issue neatly shows the outrageous level of incompetency I have reached maintaining this project which by now relies on blind trust in CI and luck 🤦♂️.
Without digging into the reasons which to me are very clear, you may read this comment as a hint at my openness to pass on the torch of maintenance.
In the mean time, things like these might keep happening unless I pace myself and prefer follow my very own rules over incorporating contributions.
Hi all,
Thanks for the fix of my mistake. I've been away on work, so not been able to get online until now.
I had expected the CI to catch import problems like that, with any test that called any function using pathlike triggering it. My fault for putting it as a forward refererence maybe.This solution is better than i would have done, which is just change os.pathlike to pathlib.Path (introduced 3.5) and see if mypy agreed.
Using the subscriptable form from 3.9+ seemed like the correct type, but i'm not sure the subscript really adds much value (it was suggested by a static checker). So it could be just os.Pathlike for 3.6+I'm hesitant to suggest adding mypy to the CI, as it would force all contributers to type everything or add type ignore comments when mypy cannot check it.
Thanks again!
Thanks for chiming in @Yobmod.
The continuation of this conversation is here: #1223 (comment)
- added a commit that references this issue
on Sep 16, 2021
This morning our builds started to fail as soon we moved to 3.1.15.
I seems that 3.1.15 makes use of os.PathLike that was introduced in python 3.6.
From README.md seems that Python 3.5 is still supported.