Commit "Rewrite local links for RST" emits "WARNING: file not found"

[jschueller]

Latest release 0.3.2 breaks the world for us as the last commit 7e79165 breaks a link to a plot directive script (here empirical_cdf-1.py):

Warning, treated as error
/tmp/openturns/python/src/sphinx_build/theory/data_analysis/empirical_cdf.rst.rst:file not found: 'theory/data_analysis/empirical_cdf-1.py'

Here is the rst file, it contains a plot directive and as well a link to a notebook:
https://github.com/openturns/openturns/blob/master/python/doc/theory/data_analysis/empirical_cdf.rst

Here is the rendered version, before the 0.3.2 update:
http://openturns.github.io/openturns/master/theory/data_analysis/empirical_cdf.html

[mgeier]

So you are saying everything works perfectly but there is a warning?

[jschueller]

No, the renderend version was built with 0.3.1, before the 0.3.2 update. We treat sphinx warnings as errors. If I believe the message emited, the pyplot script may be missing in the html.

[mgeier]

And what happens if you allow warnings?

This could be relevant: #130.

[jschueller]

If I allow warnings, the build succeeds.
The problem is that it emits a huge number of these, and that prevents us from catching the other useful warnings.

I prepared a very small example with the use of nbsphinx with a plot directive: https://github.com/jschueller/foo, see the travis log at: https://travis-ci.org/jschueller/foo.

We see that it emits 4 warnings on a single plot directive. We have lots of these...

[jschueller]

Note that after some checking the link to the script is not broken as I suggested, issue title changed.

[mgeier]

Thanks for confirming that the warning is the only problem, now we can try to solve this.

Ideas:

• just remove the warning
• create a new config option to disable this specific warning
• modify all warnings to be able to use supress_warnings, already discussed in Disable warnings for links to non-existing local files #130

Any more ideas?

Any comments?

[jschueller]

I'd prefer to actually fix the warning in nbsphinx. Do you have a rough idea of what could cause it in that commit I pointed out ?

[mgeier]

I know exactly what causes it:

nbsphinx/src/nbsphinx.py

Lines 1080 to 1083 in 89cbb76

	if not os.path.isfile(os.path.join(env.srcdir, file)):
	env.app.warn('file not found: {!r}'.format(file),
	env.doc2path(env.docname))
	continue # Link is ignored

nbsphinx was already doing this for a long time for Jupyter notebooks, but with the commit you mentioned, it started to apply the same thing to all Sphinx source files, including RST files.

Now the problem is that the "plot" directive is generating "invalid" RST code in the sense that it uses link locations that don't exist in the source directory (but they will at some point exist in the output directory).

[jschueller]

• Why do we have to touch links other than ipynbs ?
• The plot directive does not yield warnings by itself if it links to the not preexisting python script, how is this case handled ?

[mgeier]

Why do we have to touch links other than ipynbs ?

We don't have to, that's a free decision.
I thought it would be nice if links in RST files work the same as links in notebook files.
As an added benefit, those links work also when viewing the RST file on Github and other rendered RST views.
See http://nbsphinx.readthedocs.io/en/0.3.2/a-normal-rst-file.html#links-to-notebooks-and-other-sphinx-source-files

The plot directive does not yield warnings by itself if it links to the not preexisting python script, how is this case handled ?

I don't exactly understand the question, but I'm trying to answer what I think you are asking:
Normally, the RST parser doesn't check if links point to anything meaningful, they are just blindly generated.
I added a warning message in nbsphinx if a link points to a non-existing local file, because if you manually write this link in a Jupyter notebook, this is most likely a mistake, e.g. a typo.
As it turns out, when extending this to RST files, there are some cases where people would either manually or via a Sphinx extension create local links that don't point to any file in the source directory, but to a file in the output directory, which is created long after parsing the source file.

[jschueller]

This is clearer now.
Would it be possible to add an option to decide whether links should be checked against non-existing local files then ?
I found #130 is related.