Document how to test a notebook with jupytext --check

[mwouts]

We should provide a sample notebook with

• a function
• doctests
• and explicit unit tests

and document how to test it with jupytext --check [pytest].

How does that compare to

• treon,
• nbval
• or to this blog post?

[mwouts]

Apparently doctest, unittest and pytest can only run on real .py files, not on stdin. So if we want to test notebooks using jupytext --check, we'd have to write the python form to a temporary .py script, and test that one using the given command. Maybe jupytext --check could create the temp file when {} is found in the command to be executed, just like find -exec does?

[mwouts]

@mattharrison, you mentioned at #220 that you would like to run doctest on (Markdown) notebooks.

May I ask if I understand correctly that you plan to use doctest to make sure that the outputs don't change (like nbval does)? Or are you just looking for a way to make sure that the doctests inside the notebook are correct, like executing import doctest; doctest.testmod() in the notebook, but maybe outside of the notebook, like here:

jupytext --to py notebook.md # (or notebook.ipynb)
python -m doctest notebook.py

[mattharrison]

I have authored multiple books using rst. I enter my Python code as if it were typed at the terminal and then I can run doctest to ensure that my book "runs".

I'd like to be able to do similar with Jupytext, ie write a book, but using Markdown (because I want to go back and forth from Jupyter to try things out). When writing about machine learning/data analysis, having the output is critical for readers.

Here's an example from rst:

How to do it…
~~~~~~~~~~~~~


.. code:: pycon

   >>> pd.set_option('max_columns', 4, 'max_rows', 10)

.. code:: pycon

    >>> movies = pd.read_csv('data/movie.csv')
    >>> movies.head()
       color      director_name  ...  aspect_ratio  movie_facebook_likes
    0  Color      James Cameron  ...          1.78                 33000
    1  Color     Gore Verbinski  ...          2.35                     0
    2  Color         Sam Mendes  ...          2.35                 85000
    3  Color  Christopher Nolan  ...          2.35                164000
    4    NaN        Doug Walker  ...           NaN                     0
    <BLANKLINE>
    [5 rows x 28 columns]

Realize this might be "out of scope" for jupytext, but would love to be able to get the output in markdown. :)

[mwouts]

I see! Thanks for the clarification, this is an interesting use case. Since Jupytext Markdown is a format for Jupyter notebooks, maybe we'll be able to do that using notebook tools rather than doctest, but the idea is exactly the same.

Precisely, I expect that, when outputs become available in Markdown, you will be able to either

1. re-run the notebook (jupytext --execute book.md) and check e.g. with git diff if any output has changed, or
2. test the notebook with nbval (works similarly, but does not requires git, and allows to test only certain cells)

[mwouts]

Just a quick note for myself - some of the programs above do not accept files on stdin. So maybe we could implement a temp file facility in Jupytext, and allow one to call pytest like:

jupytext notebook.ipynb --check 'python -m pytest {}'