!pip not installing packages as expected

[rebornix]

I recently installed Python 3.9.6 from python.org and tried to run notebook against this interpreter but ran into a couple of issues.

Problem

• Install Python 3.9.6, which was not added to the PATH
• Open a notebook, Python 3.9.6 is listed in kernel list, select this kernel
• Run import pandas as pd, it prompts to ask to install ipykernel
• After ipykernel is installed, rerun the cell, we get error "pandas is missing"
• Insert a new cell, run !pip install pandas, it works but it installs the pandas package to python 2.7 as that the interpreter in the PATH

Proposal

The underling problem is the disconnect between Python active interpreter, Jupyter kernel, and the python interpreter in the PATH. If users are using pyenv, venv, or conda, the extension will activate the environment, which will set up the environment properly, however we don't have such thing for the vanilla Python interpreter.

We should probably

• Show a warning for !pip if the pip in the PATH is not the one in the interpreter behind the selected kernel
• Automatically add the pip from the interpreter of the selected kernel to the PATH temporarily

[rchiodo]

%pip is actually the command you should use. !pip just does a shell execute and will use the python on the path.

https://stackoverflow.com/questions/65547021/difference-between-pip-and-pip-in-python-jupyter-notebook-and-colab

[rchiodo]

Personally I think we should put up an error if the user has !pip or !conda as they don't do what the user expects.

[DonJayamanne]

Your is a regression. which pytjon used to point to the right interpreter, I fixed that, I even recall shifting this working at one point to peng.

And yes, we should use %pip

[rebornix]

Agree that we should show an error for !pip and suggest users to use %pip as a code action.

[DonJayamanne]

that we should show an error for !pip and suggest

Not sure about the error, as users might want to do that. A suggestion probably.

[greazer]

This is a common problem in the jupyter tools ecosystem all up. Consider using code-level diagnostics (squiggles, etc) to notify the user.

[DonJayamanne]

Suggestion:

• Add a diagnostic error for usage of !pip and recommend using %pip or %conda if its a conda environment.