Skip to content

Commit

Permalink
Update faq.md
Browse files Browse the repository at this point in the history
  • Loading branch information
mwouts committed Jul 2, 2019
1 parent 9d7ae13 commit 8675d07
Showing 1 changed file with 32 additions and 18 deletions.
50 changes: 32 additions & 18 deletions docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,41 +2,51 @@

## What is Jupytext?

Jupytext is a Python package that can convert Jupyter notebooks to scripts or Markdown documents. It can also convert these text documents back to Jupyter notebooks.
Jupytext is a Python package that provides _two-way_ conversion between Jupyter Notebooks and several other text-based formats like Markdown documents or scripts.

## Why would I want to convert my notebooks to text?

The text representation have much cleaner diffs than the original notebook format. Merging multiple contributions to a notebook in any of these text formats is easier than with the JSON format. Last but not least, acting on a notebook represented as text (spell check, reformat, ...) is sometimes more comfortable than in Jupyter.

## How do I use Jupytext?

Open the notebook you want to version control. _Pair_ the notebook to a script or a Markdown file using either the [Jupytext Menu](https://github.com/mwouts/jupytext/blob/master/README.md#jupytext-menu-in-jupyter-notebook) in Jupyter Notebook or the [Jupytext Commands](https://github.com/mwouts/jupytext/blob/master/README.md#jupytext-commands-in-jupyterlab) in JupyterLab.
Open the notebook that you want to version control. _Pair_ the notebook to a script or a Markdown file using either the [Jupytext Menu](https://github.com/mwouts/jupytext/blob/master/README.md#jupytext-menu-in-jupyter-notebook) in Jupyter Notebook or the [Jupytext Commands](https://github.com/mwouts/jupytext/blob/master/README.md#jupytext-commands-in-jupyterlab) in JupyterLab.

Save the notebook, and you get two copies of the notebook: the original `*.ipynb` file, together with its paired text representation.

Read more about how to use Jupytext in the [documentation](using-server.md).

## Which Jupytext format do you recommend?

I tend to use the Markdown format for notebooks that contain more text than code, as Markdown documents are conveniently edited in IDEs and also well rendered on GitHub.
Notebooks that contain more text than code are best represented as Markdown documents. These are conveniently edited in IDEs and are also well rendered on GitHub.

Saving notebooks as scripts is a convenient choice when you want to refactor your notebook in an IDE (or import it in another notebook, etc). Use the `percent` format if you prefer to get explicit cell markers (compatible with VScode, PyCharm, Spyder, Hydrogen...). If you prefer to get the minimal amount of cell markers, go for the `light` format.
Saving notebooks as scripts is an appropriate choice when you want to act on the code (refactor the code, import it in another script or notebook, etc). Use the `percent` format if you prefer to get explicit cell markers (compatible with VScode, PyCharm, Spyder, Hydrogen...). And if you prefer to get the minimal amount of cell markers, go for the `light` format.

## Can I see a sample of each format?

Go to [our demo folder](https://github.com/mwouts/jupytext/tree/master/demo) and see how our sample `World population` notebook is represented in each format.

## Can I edit the paired text file?

Yes! And when you're done, refresh the notebook in Jupyter. Refreshing will bring the latest changes to your notebook.
Yes! When you're done, reload the notebook in Jupyter. There, you will see the updated input cells combined with the matching output cells from the `.ipynb` file.

## Do I need to close my notebook in Jupyter?

No, you don't (*). You can edit the paired text file and simply refresh your navigator to reload the updated input cells. When you refresh the notebook, the kernel variables are preserved, so you can continue your work where you left it.

(*) Please read about Jupyter's autosave below.

## How do paired notebooks work?

The `.ipynb` file contains the full notebook. The paired text file only contains the input cells and selected metadata. When the notebook is loaded by Jupyter, input cells are loaded from the text file, while the output cells and the filtered metadata are restored using the `.ipynb` file.
The `.ipynb` file contains the full notebook. The paired text file only contains the input cells and selected metadata. When the notebook is loaded by Jupyter, input cells are loaded from the text file, while the output cells and the filtered metadata are restored using the `.ipynb` file. When the notebook is saved in Jupyter, the two files are updated to match the current content of the notebook.

## Can I create a notebook from a text file?

Certainly. Open your pre-existing scripts or Markdown files as notebooks with a click in Jupyter Notebook, and with the _Open as Notebook_ menu in JupyterLab. Pair these document to an `.ipynb` file if you want to see the outputs preserved when you refresh the notebook.
Certainly. Open your pre-existing scripts or Markdown files as notebooks with a click in Jupyter Notebook, and with the _Open as Notebook_ menu in JupyterLab.

If you prefer to do that programmatically, use one of
The text formats do not store the output cells. If you want to preserve these when you refresh the notebook, be sure to pair the text file to an `.ipynb` file.

If you want to convert text formats to notebooks programmatically, use one of
```bash
jupytext --to ipynb *.md # convert all .md files to notebooks with no outputs
jupytext --to ipynb --execute *.md # convert all .md files to notebooks and execute them
Expand All @@ -45,25 +55,29 @@ jupytext --set-formats ipynb,md --execute *.md # convert all .md files to paire

## Which files should I version control?

Unless you want to version control the output cells, you should version the text file only (and add `*.ipynb` to `.gitignore`). As discussed above, Jupyter will let you open the text representation as a notebook and will re-create the `.ipynb` file when you save the notebook.
Unless you want to version the outputs, you should version *only the text representation*. The paired `.ipynb` file can safely be deleted. It will be recreated locally the next time you open the notebook (from the text file) and save it.

Note that if you version control both the `.md` and `.ipynb` files, you can configure `git diff` to [ignore the diffs on the `.ipynb` files](https://github.com/mwouts/jupytext/issues/251).
Note that if you version both the `.md` and `.ipynb` files, you can configure `git diff` to [ignore the diffs on the `.ipynb` files](https://github.com/mwouts/jupytext/issues/251).

## Jupyter warns me that the file has changed on disk

By default, Jupyter saves your notebook every 2 minutes. Fortunately, it is also aware that you have edited the text file.
By default, Jupyter saves your notebook every 2 minutes. Fortunately, it is also aware that you have edited the text file, yielding this message.

You should simply click on _Reload_.

You can deactivate the autosave in Jupyter with the Jupytext Menu in Jupyter Notebook, and with the _Autosave Document_ setting in JupyterLab. See [here](https://stackoverflow.com/questions/25631344/turn-off-autosave-in-ipython-notebook/56549758#56549758) if you want to permanently deactivate the autosave in Jupyter Notebook.
Note you can deactivate Jupyter's autosave function with the Jupytext Menu in Jupyter Notebook, and with the _Autosave Document_ setting in JupyterLab. See [here](https://stackoverflow.com/questions/25631344/turn-off-autosave-in-ipython-notebook/56549758#56549758) if you want to permanently deactivate the autosave in Jupyter Notebook.

## When I refresh, Jupyter warns me that my notebook has unsaved changes
## When I reload, Jupyter warns me that my notebook has unsaved changes

Oh - you have edited both the notebook and the paired text file at the same time? Backup the text file (`git stash`), save the notebook, and merge your changes on the text file (`git stash pop`). When you're done, refresh the notebook in Jupyter.
Oh - you have edited both the notebook and the paired text file at the same time? If you know which version you want to keep, save it and reload the other. If you want to compare and merge both versions, backup the text file (with e.g. `git stash`), save the notebook, and merge the updated paired file with the backup (with e.g. `git stash pop`). Then, refresh the notebook in Jupyter.

If your IDE has the ability to compare the changes in memory versus on disk (like PyCharm), you can simply save the notebook and let your IDE do the merge.

## Jupyter complains that the `.ipynb` file is more recent than the text representation

This happens if you have edited the `.ipynb` file outside of Jupyter. Manual action is requested as the paired text representation may be outdated. Please edit (`touch`) the paired `.md` or `.py` file if it is not outdated, or if it is, delete it, or update it with
This happens if you have edited the `.ipynb` file outside of Jupyter. It is a safeguard to avoid overwriting the input cells of the notebook with an outdated text file.

Manual action is requested as the paired text representation may be outdated. Please edit (`touch`) the paired `.md` or `.py` file if it is not outdated, or if it is, delete it, or update it with
```bash
jupytext --sync notebook.ipynb
```
Expand All @@ -79,11 +93,11 @@ jupytext --set-formats ipynb,py:light notebook.ipynb # Pair a notebook to a li
jupytext --sync notebook.ipynb # Sync the two representations
```

## If only I had known of Jupytext before!
## Can I re-write my git history to use text files instead of notebooks?

Do you feel like rewriting the history of your repository and replacing every `.ipynb` file with its Jupytext Markdown representation? Technically that's just a matter of executing:
Indeed! You can substitute every `.ipynb` file in the project history with its Jupytext Markdown representation using e.g.:
```bash
git filter-branch --tree-filter 'jupytext --to md */*.ipynb && rm -f */*.ipynb' HEAD
```

See the outcome on the [Python Data Science Handbook](https://github.com/mwouts/PythonDataScienceHandbook/tree/jupytext_no_ipynb)...
See the result and the cleaner diff history in the case of the [Python Data Science Handbook](https://github.com/mwouts/PythonDataScienceHandbook/tree/jupytext_no_ipynb).

0 comments on commit 8675d07

Please sign in to comment.