Issue with installing the required Python packages (on both Windows 10 and Ubuntu 20.04)

[guidoising]

Hello Stefanie,

I have recently received my copy of the second edition of your book.
Trying to follow along, I ran into an issue an page 39 with the command: (book_env) $ pip3 install -r requirements.txt

The problem seems with "wheel" or later in the requirements file:

Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /home/guido/anaconda3/envs/book_env/bin/python /home/guido/anaconda3/envs/book_env/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpu0w3a_o8
         cwd: /tmp/pip-install-ycgjf7ri/scipy_652aaa3f60914357b64375128a803e93
    Complete output (171 lines):

I have retried this on Windows 10 with Anaconda 3 and then also on a "freh install" of Ubuntu 20.04 LTS in a virtual machine.
Please see the attached traces from both OS.

Please let me know if you require additional information.

Best regards,
Guido

Windows.txt
Ubuntu.txt

[stefmolin]

Hi @guidoising,

It looks like you are using Python 3.9, which doesn't yet have wheels for scipy and numpy. Since you are using conda, you can easily specify the version of Python to use when you create the virtual environment by adding python=3.8. You will have to create a new environment. Both packages have wheels for Python 3.8, but let me know if you still have issues.

It also looks like you hit control-c, but depending on what was already installed the wheel can fail while the packages still get installed (it falls back on building without the wheel). Once the wheel package is installed, this might not happen (I don't recall and I'm currently without access to a computer for a few days). If specifying Python 3.8 doesn't work, then you can try telling pip to ignore the wheels using this argument.

Let me know if either of those works. If not, I can look further into this when I'm back in a few days.

[guidoising]

Hi Stefanie,

many thanks for your ultra-swift reply. I have removed the initial book_env and recreated it with the option you advised:

> conda create --name book_env python=3.8

This has resolved the issues, both on Win10 and Ubuntu. The installation completed without errors!

Best regards and again many thanks for your help. Enjoy the time without access to a computer ;-)
Guido

`

[guidoising]

Hi Stefanie,

one addendum, which might prove useful for others using Windows:

After I got it working yesterday, I have decided to "clean up" my environment and perfom a "fresh install" of Anaconda 3.
When I followed the necessary steps to setup the environemnt, I received the following error message:

$ conda create --name book_env python=3.8
$ conda activate book_env
$ pip3 install -r .\requirements.txt
$ jupyter lab

ImportError: DLL load failed while importing win32api: Das angegebene Modul wurde nicht gefunden.
ImportError: DLL load failed while importing win32api: The system cannot find the file specified.

This was fixed according to xlwings/xlwings#1174 with

$ conda install -c anaconda pywin32

Testing showed that this was only necessary on my Windows 10 environment. The Ubuntu 20.04 virtual machine worked without. I assume the missing DLL was installed in the previously environment through my experiments with other packages, i.e. via installation of Jupyter Notebook/Lab through the anaconda navigator.

... just in case others might stumble as well.

Best regards,
Guido

[stefmolin]

Hi Guido,

Glad you got it working, and thanks for looking into this further. I have some additional questions for you on your Windows setup since pywin32 should be automatically installed with the dependencies for Jupyter:

1. Are you using Anaconda or Miniconda?
2. Also, if you take a look at the packages in the Anaconda installer, it has pywin32 included for Python 3.8 and 3.7. I remember you were using Python 3.9, but I don't see that as an option. Can you check if the installer you used has pywin32?

[guidoising]

Hello Stefanie,

1.) I'm using Anaconda 3 (Anaconda3-2020.11-Windows-x86_64.exe) - this is the current version that you would download from here: https://www.anaconda.com/products/individual [https://repo.anaconda.com/archive/Anaconda3-2020.11-Windows-x86_64.exe]

2.) To be honest, I have no clue how or when the Python 3.9 came into play - I must have inadvertedly installed that through Anaconda Navigator when being prompted to upgrade something and then decided to upgrade Python as well (which was obviously not a good idea in terms of compatibility)

Now, after clean up and reinstall of Anaconda, the base environment is on following versions:

  python 3.8.5
  pywin32 227
  jupyter 1.0.0
  jupyterlab 2.2.6

I have simulated the end-to-end installation process on a clean Windows 10 environment running on VirtualBox and using a freshly installed Windows 10 (Windows 10 20H2.iso).

When I create the book_env with:

conda create --name book_env python= 3.8

the book_env has the following release: python 3.8.8. There is no pywin32, jupyter or jupyterlab yet.

I need to install git for windows (Git-2.31.1-64-bit.exe). After

(book_env) PS C:\Users\guido> git clone https://github.com/stefmolin/Hands-On-Data-Analysis-with-Pandas-2nd-edition.git

(book_env) PS C:\Users\guido> cd Hands-On-Data-Analysis-with-Pandas-2nd-edition

(book_env) PS C:\Users\guido\Hands-On-Data-Analysis-with-Pandas-2nd-edition> pip3 install -r .\requirements.txt

Completes successfully. Versions are:

  python 3.8.8
  pywin32 300
  jupyter 1.0.0
  jupyterlab 3.0.4

However, trying to run jupyter lab results in error:

When I then execute

(book_env) PS C:\Users\guido\Hands-On-Data-Analysis-with-Pandas-2nd-edition> conda install -c anaconda pywin32

The following indicates, that pywin32 version 227 is installed (instead of 300)

However, when verifying the pywin32 version, it still is 300! But now jupyter lab is running!

... That was a very long answer to a short question, my apologies! I do not mean to waste your time - but being atomic physicist by trade, I had to research the issue under "lab conditions". The traces show that

a) pywin32 is installed in version 300 together with python 3.8.8 but this does not help jupyter lab to work in itself
b) The installation "conda install -c anaconda pywin32 magically seems to fix the issue, although it apparently does not change the version of pywin32
c) After the installation, jupyter labs works - but we don't fully understand why.

Maybe you can read more from the traces - but don't spend to much time.
This may be an effect of "electron psychology" as my old Physics teacher would have suspected ;-)

Please let me know if there is anything else you need to have simulated or traced - happy to do so if it helps.

Best regards,
Guido

[stefmolin]

Hi Guido,

Thank you for your detailed response. From my initial research, it does seem to be an issue with the installation of the pywin32 package and is still a bug in the latest version of Anaconda (as per this comment on a similar issue). I also remember seeing something about why the command you have in b) fixes the issue: there is a setup bug, but the anaconda command runs a post install script to fix it.

I don't think we need to look into it any further since we know a fix that works consistently -- thanks for testing again! I pinned this issue for others to reference :)