Improve documentation about `%load_ext kedro.ipython` to make it more visible

[astrojuanlu]

Description

We should make %load_ext kedro.ipython more visible, since it's tool-independent (as long as it's IPython-compatible) and also independent on how the service was launched (think for instance Databricks notebooks, Google Colab, and VSCode).

Context

A user wanted to use the Kedro extension on VSCode and didn't find a way in the docs https://www.linen.dev/s/kedro/t/13090672/hi-everyone-is-there-some-setting-that-could-allow-for-kedro#30d9d3e7-73e3-4719-8be5-882a947088e4 cc @m-gris

even if it's there already https://docs.kedro.org/en/stable/notebooks_and_ipython/kedro_and_notebooks.html#managed-services.

In fact, I don't even use kedro jupyter notebook that much 🤔 always resort to %load_ext kedro.ipython everywhere.

Possible Implementation

We should probably restructure those pages a bit, so that it's clear that the command can be used in many places, and not just "managed services".

Possible Alternatives

I didn't think any.

[astrojuanlu]

I almost open this issue again 🙃

Another user got confused because they couldn't really use kedro jupyter notebook/lab in their environment, and %load_ext kedro.ipython worked perfectly, but I had to show them the solution:

https://linen-slack.kedro.org/t/15706050/hi-everyone-i-am-getting-acquainted-with-kedro-and-i-want-to#51630c54-7080-4cc7-bc5a-4cdba1d1cda9

[astrojuanlu]

For context, kedro jupyter notebook creates a kernel and launches Jupyter with it

kedro/kedro/framework/cli/jupyter.py

Lines 90 to 91 in c3c93cb

	"jupyter",
	["notebook", f"--MultiKernelManager.default_kernel_name={kernel_name}"]

[astrojuanlu]

@noklam mentions that kedro jupyter setup works so-so on Databricks, Amazon Sagemaker and so on.

[stichbury]

kedro jupyter notebook is clearly a Kedro command in that it's formatted in a similar way to all our others

%load_ext kedro.ipython is a weird looking thing that isn't easy to parse, remember or understand. I'm unconvinced on how to document this at first sight and I think we instead need to be documenting the problem that it solves. As in "How do I do X?" and the answer "You do it with load_ext kedro.ipython" and be really clear which section of the docs it goes into.

I'm happy to do this but need more steer about (a) the problem and (b) the location in docs that we include it.

[astrojuanlu]

As in "How do I do X?" and the answer "You do it with load_ext kedro.ipython" and be really clear which section of the docs it goes into.

💯 %

%load_ext kedro.ipython, kedro jupyter notebook and kedro jupyter init all solve the same problem: allowing users to quickly load a Kedro project from a Python interactive interpreter. The first one is an IPython command https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-load_ext the other two are, as you say, Kedro commands.

The problem of the latter two is that they're too rigid:

• They assume you can launch your own Jupyter server, which doesn't hold on managed services (Databricks, Google Colab, JupyterHub)
• They assume you know what a Jupyter kernel is, and that you know how to choose one (most users don't know the former, and blindlessly pick whatever comes first)
• They assume you want to use a Kedro kernel, which might not be true (there are other interesting kernels that can be used with Kedro, for example ipyflow)

So I think %load_ext kedro.ipython should be at the very top of https://docs.kedro.org/en/stable/notebooks_and_ipython/kedro_and_notebooks.html, and then alternative workflows should be explained as a way to "not even have to %load_ext".

[stichbury]

See also comment in this thread: https://linen-slack.kedro.org/t/16320811/i-just-learned-that-the-command-kedro-jupyter-notebook-can-b#7a074598-5ddb-457f-ad73-d4a949158725

[astrojuanlu]

Another user whose problems with kedro jupyter were solved by using %load_ext kedro.ipython https://linen-slack.kedro.org/t/16331010/hi-sometimes-i-have-trouble-running-kedro-jupyter-notebook-i#50ac0562-8330-4697-982e-2892e2127c47

[noklam]

Almost duplicate the issue. I agree with everything said here. Ideally the Kedro specific kernel is a nice idea, in practices it suffers from different issue.

1. User don't select the correct kernel, clicking the wrong kernel on Jupyter/VSCode notebook
2. Virtual environment / Jupyter Kernel confusion - at the end the difference is subtle and people just assume they are the same. When it's not, it surprise people in a bad way.
3. Cross platform support for kernel seems to be hard - Make sure Kedro custom Jupyter Kernel works on platform like SageMaker #2412