Add documentation on ipython debugging (#3579)

* Add first draft Signed-off-by: lrcouto <laurarccouto@gmail.com> * Remoe outdated kedro jupyter convert docs Signed-off-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com> * Suggestion: Review edits Signed-off-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com> * Update FAQs Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Edit jupyter ipython debug section Signed-off-by: lrcouto <laurarccouto@gmail.com> * Change link to section that does not exist anymore Signed-off-by: L. R. Couto <laurarccouto@gmail.com> * Change link to section that does not exist anymore Signed-off-by: L. R. Couto <laurarccouto@gmail.com> * Change wording and formatting Signed-off-by: lrcouto <laurarccouto@gmail.com> * Lint Signed-off-by: lrcouto <laurarccouto@gmail.com> * Update docs/source/notebooks_and_ipython/kedro_and_notebooks.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com> * Update docs/source/notebooks_and_ipython/kedro_and_notebooks.md Co-authored-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com> Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com> * Changes to the wording, remove unnecessary section Signed-off-by: lrcouto <laurarccouto@gmail.com> * Update docs/source/notebooks_and_ipython/kedro_and_notebooks.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com> * Update docs/source/notebooks_and_ipython/kedro_and_notebooks.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com> * Update docs/source/notebooks_and_ipython/kedro_and_notebooks.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com> * Update docs/source/notebooks_and_ipython/kedro_and_notebooks.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com> * Update docs/source/notebooks_and_ipython/kedro_and_notebooks.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com> * Change passive tense on debugging section Signed-off-by: lrcouto <laurarccouto@gmail.com> --------- Signed-off-by: lrcouto <laurarccouto@gmail.com> Signed-off-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com> Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> Signed-off-by: L. R. Couto <laurarccouto@gmail.com> Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com> Co-authored-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com> Co-authored-by: Ahdra Merali <ahdra.merali@quantumblack.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
kedro-org · Feb 9, 2024 · f54c6fb · f54c6fb
1 parent 64296cf
commit f54c6fb
Show file tree

Hide file tree

Showing 4 changed files with 47 additions and 18 deletions.
diff --git a/docs/source/faq/faq.md b/docs/source/faq/faq.md
@@ -14,7 +14,7 @@ This is a growing set of technical FAQs. The [product FAQs on the Kedro website]
 
 ## Working with Jupyter
 
-* [How can I convert functions from Jupyter Notebooks into Kedro nodes](../notebooks_and_ipython/kedro_and_notebooks.md#how-to-use-tags-to-convert-functions-from-jupyter-notebooks-into-kedro-nodes)?
+* [How can I debug a Kedro project in a Jupyter notebook](../notebooks_and_ipython/kedro_and_notebooks.md#debugging-with-debug-and-pdb)?
 * [How do I connect a Kedro project kernel to other Jupyter clients like JupyterLab](../notebooks_and_ipython/kedro_and_notebooks.md#ipython-jupyterlab-and-other-jupyter-clients)?
 
 ## Kedro project development

diff --git a/docs/source/meta/images/jupyter_ipython_debug_command.gif b/docs/source/meta/images/jupyter_ipython_debug_command.gif
diff --git a/docs/source/meta/images/jupyter_ipython_pdb_command.gif b/docs/source/meta/images/jupyter_ipython_pdb_command.gif
diff --git a/docs/source/notebooks_and_ipython/kedro_and_notebooks.md b/docs/source/notebooks_and_ipython/kedro_and_notebooks.md
@@ -209,32 +209,61 @@ You don't need to restart the kernel for the `catalog`, `context`, `pipelines` a
 
 For more details, run `%reload_kedro?`.
 
-## How to use tags to convert functions from Jupyter notebooks into Kedro nodes
+## Useful to know (for advanced users)
+Each Kedro project has its own Jupyter kernel so you can switch between Kedro projects from a single Jupyter instance by selecting the appropriate kernel.
 
-You can use the notebook to write experimental code for your Kedro project. If you later want to convert functions you've written to Kedro nodes, you can do this using `node` tags to export them to a Python file. Say you have the following code in your notebook:
+To ensure that a Jupyter kernel always points to the correct Python executable, if one already exists with the same name `kedro_<package_name>`, then it is replaced.
 
-```ipython
-def some_action():
-    print("This function came from `notebooks/my_notebook.ipynb`")
-```
+You can use the `jupyter kernelspec` set of commands to manage your Jupyter kernels. For example, to remove a kernel, run `jupyter kernelspec remove <kernel_name>`.
 
-1. Enable tags toolbar: `View` menu -> `Cell Toolbar` -> `Tags`
-![Enable the tags toolbar graphic](../meta/images/jupyter_notebook_workflow_activating_tags.png)
+### Debugging with %debug and %pdb
 
-2. Add the `node` tag to the cell containing your function
-![Add the node tag graphic](../meta/images/jupyter_notebook_workflow_tagging_nodes.png)
+ You can use the `%debug` [line magic](https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-debug) to launch an interactive debugger in your Jupyter notebook. Declare it before a single-line statement to step through the execution in debug mode. You can use the argument `--breakpoint` or `-b` to provide a breakpoint.
+The follow sequence occurs when `%debug` runs immediately after an error occurs:
+ - The stack trace of the last unhandled exception loads.
+ - The program stops at the point where the exception occurred.
+ - An interactive shell where the user can navigate through the stack trace opens.
 
+ You can then inspect the value of expressions and arguments, or add breakpoints to the code.
 
-3. Save your Jupyter notebook to `notebooks/my_notebook.ipynb`
-4. From your terminal, run `kedro jupyter convert notebooks/my_notebook.ipynb` from the Kedro project directory. The output is a Python file `src/<package_name>/nodes/my_notebook.py` containing the `some_action` function definition
-5. The `some_action` function can now be used in your Kedro pipelines
+<details>
+<summary>Click to see an example.</summary>
 
-## Useful to know (for advanced users)
-Each Kedro project has its own Jupyter kernel so you can switch between Kedro projects from a single Jupyter instance by selecting the appropriate kernel.
+![jupyter_ipython_debug_command](../meta/images/jupyter_ipython_debug_command.gif)
 
-If a Jupyter kernel with the name `kedro_<package_name>` already exists then it is replaced. This ensures that the kernel always points to the correct Python executable. For example, if you change conda environment in a Kedro project then you should re-run `kedro jupyter notebook` to replace the kernel specification with one that points to the new environment.
+</details>
+
+---
+
+You can set up the debugger to run automatically when an exception occurs by using the `%pdb` [line magic](https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-pdb). This automatic behaviour can be enabled with `%pdb 1` or `%pdb on` before executing a program, and disabled with `%pdb 0` or `%pdb off`.
+
+<details>
+<summary>Click to see an example.</summary>
+
+ ![jupyter_ipython_pdb_command](../meta/images/jupyter_ipython_pdb_command.gif)
+
+</details>
+
+---
+
+Some examples of the possible commands that can be used to interact with the ipdb shell are as follows:
+
+| Command           | Description                                           |
+| ----------------- | ----------------------------------------------------- |
+| `list`            | Show the current location in the file                 |
+| `h(elp)`          | Show a list of commands, or find help on a specific command |
+| `q(uit)`          | Quit the debugger and the program                     |
+| `c(ontinue)`      | Quit the debugger, continue in the program             |
+| `n(ext)`          | Go to the next step of the program                     |
+| `<enter>`         | Repeat the previous command                            |
+| `p(rint)`         | Print variables                                       |
+| `s(tep)`          | Step into a subroutine                                |
+| `r(eturn)`        | Return out of a subroutine                            |
+| `b(reak)`         | Insert a breakpoint                                   |
+| `a(rgs)`          | Print the argument list of the current function        |
+
+For more information, use the `help` command in the debugger, or take at the [ipdb repository](https://github.com/gotcha/ipdb) for guidance.
 
-You can use the `jupyter kernelspec` set of commands to manage your Jupyter kernels. For example, to remove a kernel, run `jupyter kernelspec remove <kernel_name>`.
 
 ### Managed services