python3.9 get-pip.py results in "AttributeError: 'HTMLParser' object has no attribute 'unescape'"

[msoultan]

• pip version:
  version 20.2.4

• Python version:
  Python 3.9

• Operating system:
  Ubuntu 18.04.5 LTS

When I try an install pip I get the following error:

$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1842k  100 1842k    0     0  4168k      0 --:--:-- --:--:-- --:--:-- 4159k
$ python3.9 get-pip.py --user
Traceback (most recent call last):
  File "/tmp/get-pip.py", line 23704, in <module>
    main()
  File "/tmp/get-pip.py", line 198, in main
    bootstrap(tmpdir=tmpdir)
  File "/tmp/get-pip.py", line 120, in bootstrap
    import setuptools  # noqa
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 14, in <module>
    from setuptools.dist import Distribution, Feature
  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 24, in <module>
    from setuptools.depends import Require
  File "/usr/lib/python3/dist-packages/setuptools/depends.py", line 7, in <module>
    from .py33compat import Bytecode
  File "/usr/lib/python3/dist-packages/setuptools/py33compat.py", line 54, in <module>
    unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
AttributeError: 'HTMLParser' object has no attribute 'unescape'

[uranusjr]

Do you have a file named html.py or directory html in your current directory? You need to (re)move it since it is clashing with the built-in html module.

[uranusjr]

Ah, I think I know what’s going on. PR soon.

[msoultan]

And in case it helps, here's exactly what I'm doing:

********@*************:/tmp$ mkdir test

********@*************:/tmp$ cd test

********@*************:/tmp/test$ ls -l
total 0

********@*************:/tmp/test$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1842k  100 1842k    0     0  2606k      0 --:--:-- --:--:-- --:--:-- 2606k

********@*************:/tmp/test$ ls -l
total 1844
-rw-rw-r-- 1 ******** ******** 1886796 Nov 13 10:55 get-pip.py

********@*************:/tmp/test$ python3.9 get-pip.py --user
Traceback (most recent call last):
  File "/tmp/test/get-pip.py", line 23704, in <module>
    main()
  File "/tmp/test/get-pip.py", line 198, in main
    bootstrap(tmpdir=tmpdir)
  File "/tmp/test/get-pip.py", line 120, in bootstrap
    import setuptools  # noqa
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 14, in <module>
    from setuptools.dist import Distribution, Feature
  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 24, in <module>
    from setuptools.depends import Require
  File "/usr/lib/python3/dist-packages/setuptools/depends.py", line 7, in <module>
    from .py33compat import Bytecode
  File "/usr/lib/python3/dist-packages/setuptools/py33compat.py", line 54, in <module>
    unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
AttributeError: 'HTMLParser' object has no attribute 'unescape'

[uranusjr]

BTW why do you need get-pip on 3.9 in the first place? Modern Python versions have a built-in ensurepip module that covers the same functionality: python3.9 -m ensurepip --user.

[msoultan]

I'm just following the instructions to install pip for Python 3.9 as described here:

https://pip.pypa.io/en/stable/installing/

[uranusjr]

Ah, I think I know what’s going on. PR soon.

Turns out this has been fixed for at long time: pypa/setuptools@53b8db3

But your setuptools package is ancient and does not contain it.

Judging from your paths, you seem to be on a Debian-derived system. How did you install Python 3.9? You should use apt to install pip if you installed Python from apt, instead of using get-pip.py, which may break your system (as described in the warning).

[msoultan]

So to install Python 3.9 on Ubuntu 16 I use the following commands:

sudo apt install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.9

Is this not the correct way to install python3.9? Is there a better way to do it? I used to use apt to install pip, but it was always installing an older version - I haven't tried it recently.

[uranusjr]

It is a correct way to install Python 3.9, and should already contain pip. Does python3.9 -m pip --version prints the installation information?

It seems like someone reported a similar issue to deadsnakes recently: deadsnakes/issues#117. You probably want to join the discussion there since it would be much easier to reach deadsnakes maintainers.

[msoultan]

Hmm... doesn't seem so:

$ python3.9 -m pip --version
/usr/bin/python3.9: No module named pip

[pfmoore]

Then that's a bug in how Ubuntu package Python. I have a feeling it's a known problem - I vaguely recall that Ubuntu split pip out from the "main" Python package - but if so, you need to do whatever's required in Ubuntu to get a complete Python installation (which means it includes pip).

[msoultan]

So, the recommendation posted to the ticket I created in deadsnakes (referenced in this ticket) was to use sudo apt-get install python3-pip which I thought was no-longer to be used because it installed an old version so I switched to get-pip.py. That being said, I didn't know that I probably should have just run python3.9 -m pip install pip --upgrade in order to bring my python3.9 pip version up-to-date. So now I guess the right thing to do is use python3-pip instead of get-pip.py? That being said, the person in the other thread on deadsnakes said that this issue that I mentioned in this thread is a bug - not sure what to do other than report this stuff because I'm far from an expert on this!

One side-effect of this is that it seems like I don't get a pip3.9 executable - which I guess really isn't the end of the world as it seems like the general recommendation is to use python3.9 -m pip install ... moving forward.

Also, performing python3.9's pip update overwrites pip and pip3 to point to python3.9, which doesn't seem like a good idea for system-level scripts that use those with the assumption that pip -> python2 and pip3 -> system-level python version (i.e. 3.6 for Ubuntu 16). I'm not sure if that's a problem, but it does seem weird that pip/pip3 are changed.