Opening non Notebook related scripts automatically adds metadata

[erdnaavlis]

Imagine I have a non-notebook related script which contents are:

print('Hello World')

If I open in it using the jupyter interface, after installing jupytext, it opens as if it were a notebook. Then if I do some minor edits and save it, its contents become:

# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.3'
#       jupytext_version: 0.8.2
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
#   language_info:
#     codemirror_mode:
#       name: ipython
#       version: 3
#     file_extension: .py
#     mimetype: text/x-python
#     name: python
#     nbconvert_exporter: python
#     pygments_lexer: ipython3
#     version: 3.7.0
#   toc:
#     nav_menu: {}
#     number_sections: true
#     sideBar: true
#     skip_h1_title: false
#     toc_cell: false
#     toc_position: {}
#     toc_section_display: block
#     toc_window_display: false
#   varInspector:
#     cols:
#       lenName: 16
#       lenType: 16
#       lenVar: 40
#     kernels_config:
#       python:
#         delete_cmd_postfix: ''
#         delete_cmd_prefix: 'del '
#         library: var_list.py
#         varRefreshCmd: print(var_dic_list())
#       r:
#         delete_cmd_postfix: ') '
#         delete_cmd_prefix: rm(
#         library: var_list.r
#         varRefreshCmd: 'cat(var_dic_list()) '
#     types_to_exclude:
#     - module
#     - function
#     - builtin_function_or_method
#     - instance
#     - _Feature
#     window_display: false
# ---

print('Hello World!')

Is there a way to avoid jupytext interfering with native python scripts?

[mwouts]

Hello @andrethrill ! Well, currently there is no such option. But I would like to offer one! What would you think of a jupytext: noheader: true metadata that

• jupytext would insert (in the ipynb representation only) on notebooks loaded from scripts that do not have a header
• and that the user could remove or introduce manually (by editing the notebook metadata) when they want or not a header in the text representation?

[mwouts]

Your example is also a good example for #105: the YAML header is way too long, and such long headers typically introduce irrelevant noise in version control.

[erdnaavlis]

Thanks for your answer.

What would you think of a jupytext: noheader: true metadata that

• jupytext would insert (in the ipynb representation only) on notebooks loaded from scripts that do not have a header
• and that the user could remove or introduce manually (by editing the notebook metadata) when they want or not a header in the text representation?

Just to be sure we are talking about the same thing: With that approach, if the user opens a native python script, does not do any action with jupytext, does his python edits and hit save, only the python script gets saved (no .ipynb) without any extra metadata contents (no header and no cell metadata)? Otherwise, if the user changes that new parameter to False then it will become a regular notebook and both .py and .ipynb are saved? If the answers are yes to both than it sounds perfect. But I'm not sure this is exactly what you are suggesting?

Thinking differently: I don't know if it's feasible because it requires reading the file in advance but what about, if the user opens a script that has no jupytext metadata (=native python script) the regular jupyter editor fires up? This way, Not interfering with the default jupyter behaviour.

[mwouts]

Well, @andrethrill , it's a good thing that you describe very precisely your expectations!

• the noheader (or possibly header) parameter would just act on the notebook metadata, not on the cell metadata.
• we could have, and should have another parameter for determining which cell metadata should go to the text representation, cf. your other issue Add option to Ignore certain cells metadata #106
• do you expect that Jupyter and Jupytext should behave differently? Should a notebook created with command line jupytext have the header when exported to a script again? I would think the two ways of creating an ipynb notebook from a script should give the same notebook.
• I like you reformulation. Sure, opening the script with the standard editor makes all this simpler! A few people already asked for being allowed to open scripts with the text editor in jupyter notebook, see for instance this comment and the corresponding workaround (my answer). Note that this is the default in Jupyter Lab (scripts can be opened as notebooks using a context menu, see the README). For Jupyter Notebook, we could easily have a parameter on Jupytext's content manager to avoid classifying scripts as notebooks (classification of notebooks/non notebooks is done here).

[mwouts]

Hello @andrethrill , I have started working on this in the v0.8.4 branch. Jupytext will set and use the following metadata to filter header or cell metadata:

• jupytext.metadata.notebook (implemented, to be tested, see also Request: minified header #105 )
• jupytext.metadata.cell (not active yet)

[erdnaavlis]

Hi @mwouts,

Thanks for your support.

do you expect that Jupyter and Jupytext should behave differently? Should a notebook created with command line jupytext have the header when exported to a script again? I would think the two ways of creating an ipynb notebook from a script should give the same notebook.

As far as my understanding goes, I believe the behaviour should be the same from jupyter and jupytext.

Note that this is the default in Jupyter Lab (scripts can be opened as notebooks using a context menu, see the README). For Jupyter Notebook, we could easily have a parameter on Jupytext's content manager to avoid classifying scripts as notebooks (classification of notebooks/non notebooks is done here).

This makes sense to me. The reasoning behind me opening this issue was seeing if it was possible to make jupytext as less intrusive (read, not changing default behaviour) as possible in functionalities (such as editing pure python scripts) that go outside of Jupytext core (notebooks).

At the same time I also see the option of editing script as notebooks with good eyes :)

Hello @andrethrill , I have started working on this in the v0.8.4 branch.

That's awesome. Thank you so much once again for your support!

[mwouts]

Hello @andrethrill , I have prepare a pre-release version with this feature, available with

pip install jupytext==0.8.4rc1

The documentation for the metadata filtering is here.

Could you please test, and let me know if that works well for you. Comments on the documentation may also be useful !

[mwouts]

Available in v0.8.4. Note that in that version the option name has changed! It is now named freeze_metadata. And implemented both in the command line tool, and in the content manager. Thanks @andrethrill for reporting this use case!

[erdnaavlis]

@mwouts I had the chance to give it a try now. So far it looks awesome! It fits perfectly the use of my team. Thank you for your support :)