toctree contains reference to nonexisting document

[amueller]

I'm still trying to migrate scikit-learn to numpydoc master.
I'm having some trouble with this issue:
http://stackoverflow.com/questions/12206334/sphinx-autosummary-toctree-contains-reference-to-nonexisting-document-warnings

Apparently something goes wrong with when autosummary is called?
That didn't use to be the case.
I can't really do numpydoc_show_class_members = False because then there is no method table any more :(

[amueller]

so setting class_members_toctree = False solves my problem. But there still seems to be a bug...

[jorisvandenbossche]

For pandas we used the same 'hack' as in the numpy docs: https://raw.githubusercontent.com/numpy/numpy/master/doc/source/_templates/autosummary/class.rst, but I am not sure anymore if this was to solve those error messages (long time ago I did this for the pandas docs)

[amueller]

huh, we didn't need this hack, the numpydoc_show_class_members = True combine with class_members_toctree = False seems to produce the right results. It just seems kinda odd if there is only one specific configuration that is needed to make it actually work... well better than a hack.

[stefanv]

That is weird; if you figure out what's going on, let's put a note in the README at least.

[amueller]

I'm kinda punting on it for now. This configuration Works For Me™.

[jnothman]

So is this suggesting that the .. autosummary in numpydoc-mangled docstrings simply isn't working for autogeneration, hence the need for the above hacks?

This is very believable to me: I can't see anywhere in autosummary where it sees the mangled docs. It's either processing through the autosummary_generate option, at a builder-inited hook, which operates over paths, or autogen is run manually again on paths.

Now we no longer use .. autosummary for display of attributes, only methods, so we should rid of the hack use of autosummary from attribute lists, then. I'm not sure if class_members_toctree is having useful effect for method listing? (Or is it possible someone's been using numpydoc directly as a pre-process rather than as an autodoc-process-docstring hook?)

[jnothman]

So I think we basically need to recommend a hack at the template level, and that class_members_toctree should then match what the hack does :\

[mattip]

I am willing to work on this, but as I am new to the codebase would need some help parsing @jnothman 's comment above. Is there a general design document that describes what autosummary does and when? It seems there are three options for generating summaries:

• the autosummary_generate option (I guess this is what numpy does since we have autosummary_generate = glob.glob("reference/*.rst") in conf.py)
• a builder-initiated hook which operates over paths
• autogen is run manually again on paths

What do the second and third one mean? Should one use be preferred over the others?

Then there is the sentence below, what is meant by "the hack use" and what should replace it?

Now we no longer use .. autosummary for display of attributes, only methods, so we should rid of the hack use of autosummary from attribute lists

[mattip]

FWIW, adding these lines to the conf.py does not silence the warnings in numpy

numpydoc_show_class_members = True
class_members_toctree = False

[jnothman]

I'm not really sure what to say here, and my recall is weak. The different ways to run autosummary are not really relevant as long as they only collect names to autodoc from reST files, rather than from autodoc-generated reST. I suppose you could consider fixing that in autodoc. Otherwise I'm not sure how to go about fixing this.

[mattip]

TL;DR - Removing the .. HACK directive which wraps the .. autosummary directive for class attributes in the autosummary class template causes class attributes to be displayed and the warnings to disappear, with no changes needed to NumPy's conf.py

Here is what I have so far based on observations and trying out different alternatives. NumPy has a template in doc/source/_templates/autosummary/class.rst with nested directives. This template is filled in by the autosummary generator with class methods and attributes, and documentation of those methods and attributes is generated, one file for each method and attribute. Then an additional pass over the documentation is done to fill in TocTree contents. Class methods are properly added to the class documentation, but class attributes are not. If I remove the nested nature of the directive for the attributes, the class attributes now appear in the class summary and the warning is not generated.

So apparently at some point in time class methods and attributes were treated the same, and then something changed and class methods are now still added to the class summary but class attributes are not. See for instance how numpy.broadcast has a "Methods" section, but no "Attributes".

Beyond changing the template to properly index class attributes, is there something more that should be done in numpydocs to document or change this behavior?

[mattip]

Removing the nested directive from the template for class methods results in the method having two links in the output document where the attribute has only one, so something is different in the handling of class methods and attributes.

[mattip]

maybe relate to #74 ?

[mattip]

Nope, my "fix" is garbage. The attributes show up in the currently rendered documents, but formatted badly and above the methods. It seems the nested directive hack is needed to prevent duplicate rendering of both the Methods and the Attributes and that something else is happening.

[jnothman]

Formatted "badly" is a matter of taste. They're formatted just like Parameters and Returns. I.e. they're formatted as numpydoc formats things, rather than according to the decisions of autosummary.

[mattip]

Yes, sorry, I let my frustration through. My fault for not noticing the Attributes do appear in the rendered document, but formatted differently than the Methods.

[jorisvandenbossche]

It's related to the longish discussion I had here #106 (comment).
I still think it would be good if numpydoc would want to support this usecase of keeping the Attributes listing as they were before for downstream users as pandas (and numpy? don't know if they want to keep it)

I have a PR that does that (#161), but of course, that's still a bit independent from the discussion in this issue about the hack in the class template and the warnings (for this, in pandas we long time ago just copied the hack from numpy). I would also need to look at it again to again understand why it was needed.

[mattip]

Maybe I have made progress. I created templates for methods, members,and attributes under autosummary, copying the base.rst but adding a top line :orphan:. While not impacting the output (as far as I can see) it does quiet the warning. So here is what I have so far:

• The hack in the class.rst template is needed to prevent the Methods and Attributes sections from appearing twice in the class rendered document.
• The hack also causes the members and attributes label to turn into links (and generates corresponding pages). See for instance the page for numpy.broadcast
• Class members that are neither attributes nor methods are listed with the attributes, but are not rendered as links.
• A class method page is rendered with a sidebar and breadcrumbs across the top
• A class atrribute page is rendered with a truncated breadcrumb and no sidebar
• A class member page is created, even though it has no link from the parent class page. It appears in the search results.

I still do not understand why the class.rst template hack generates warning NumPy but not for SciPy . Both use exactly the same template and have very similar conf.py files, although with slightly different extension modules.

[mattip]

Could someone with commit permissions reopen since the core issue (hack needed in class template, which then causes many warnings) is not yet fixed

[rgommers]

wasn't the point of that PR to remove the warnings? not sure I understand your comment correctly, but if it's about tracking orphan so it can be removed again later, I think that's better as a new issue

[mattip]

Ok, sorry for the noise. Close this and I will open a new one.

[rgommers]

Ah no, my apologies - I thought we were in numpy/numpy#11316. NumPy has a workaround now, but numpydoc still needs a proper fix.

[ReenigneArcher]

Just came across these warnings. Not sure if there's a better way, but I solved it by manually adding the Methods heading to my class docstring. Kind of redundant, but at least docs build without the warnings now.

https://stackoverflow.com/a/77588774/11214013

Docs actually built fine before, except the warnings generated... but readthedocs builds are set to fail on any warnings which I think is a good practice to keep in place.