Theme cannot build with man page outputs

[kloczek]

First of all currently it is not possible to use straight sphinx-build command to build documentation out of source tree

+ /usr/bin/sphinx-build -n -T -b man docs build/sphinx/man
Running Sphinx v5.1.1

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 347, in eval_config_file
    exec(code, namespace)
  File "/home/tkloczko/rpmbuild/BUILD/pydata-sphinx-theme-0.10.1/docs/conf.py", line 14, in <module>
    import pydata_sphinx_theme
ModuleNotFoundError: No module named 'pydata_sphinx_theme'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/cmd/build.py", line 272, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 201, in __init__
    self.config = Config.read(self.confdir, confoverrides or {}, self.tags)
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 169, in read
    namespace = eval_config_file(filename, tags)
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 360, in eval_config_file
    raise ConfigError(msg % traceback.format_exc()) from exc
sphinx.errors.ConfigError: There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 347, in eval_config_file
    exec(code, namespace)
  File "/home/tkloczko/rpmbuild/BUILD/pydata-sphinx-theme-0.10.1/docs/conf.py", line 14, in <module>
    import pydata_sphinx_theme
ModuleNotFoundError: No module named 'pydata_sphinx_theme'

This can be fixed by patch like below:

--- a/docs/conf.py
+++ b/docs/conf.py
@@ -1,6 +1,7 @@
 # -- Path setup --------------------------------------------------------------
 import os
 import sys
+sys.path.insert(0, os.path.abspath("../src"))

 sys.path.append("scripts")
 from gallery_directive import GalleryDirective

This patch fixes what is in the comment and that can of fix is suggested in sphinx example copy.py https://www.sphinx-doc.org/en/master/usage/configuration.html#example-of-configuration-file

Than with that patch `sphinx-build fails with call trace

+ /usr/bin/sphinx-build -n -T -b man docs build/sphinx/man
Running Sphinx v5.1.1
making output directory... done
[autosummary] generating autosummary for: community/contributors.md, community/index.md, community/inspiration.md, community/manual.md, community/policies.md, community/setup.md, community/structure.md, community/topics.md, examples/api.rst, examples/blog/post1.md, ..., user_guide/navigation.rst, user_guide/page-toc.rst, user_guide/performance.rst, user_guide/readthedocs.md, user_guide/search.rst, user_guide/source-buttons.rst, user_guide/styling.rst, user_guide/theme-elements.md, user_guide/version-dropdown.rst, user_guide/web-components.rst
[autosummary] generating autosummary for: /home/tkloczko/rpmbuild/BUILD/pydata-sphinx-theme-0.10.1/docs/examples/generated/pandas.DataFrame.drop.rst, /home/tkloczko/rpmbuild/BUILD/pydata-sphinx-theme-0.10.1/docs/examples/generated/pandas.DataFrame.groupby.rst, /home/tkloczko/rpmbuild/BUILD/pydata-sphinx-theme-0.10.1/docs/examples/generated/pandas.Series.array.rst
myst v0.18.0: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=['colon_fence', 'substitution'], disable_syntax=[], all_links_external=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, highlight_code_blocks=True, number_code_blocks=[], title_to_header=False, heading_anchors=2, heading_slug_func=None, footnote_transition=True, words_per_minute=200, sub_delimiters=('{', '}'), linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area')
myst-nb v0.16.0: NbParserConfig(custom_formats={}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, execution_mode='auto', execution_cache_path='', execution_excludepatterns=(), execution_timeout=30, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_streams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_markdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False)
Using jupyter-cache at: /home/tkloczko/rpmbuild/BUILD/pydata-sphinx-theme-0.10.1/build/sphinx/.jupyter_cache
building [mo]: targets for 0 po files that are out of date
building [man]: all manpages
updating environment: [new config] 63 added, 0 changed, 0 removed
sphinx-sitemap warning: No pages generated for sitemap.xml

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/events.py", line 94, in emit
    results.append(listener.handler(self.app, *args))
  File "/usr/lib/python3.8/site-packages/ablog/post.py", line 416, in process_posts
    stdlabel = env.intersphinx_inventory.setdefault("std:label", {})  # NOQA
AttributeError: 'BuildEnvironment' object has no attribute 'intersphinx_inventory'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/cmd/build.py", line 277, in build_main
    app.build(args.force_all, filenames)
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 349, in build
    self.builder.build_update()
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 298, in build_update
    self.build(['__all__'], to_build)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 317, in build
    updated_docnames = set(self.read())
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 424, in read
    self._read_serial(docnames)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 445, in _read_serial
    self.read_doc(docname)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 498, in read_doc
    publisher.publish()
  File "/usr/lib/python3.8/site-packages/docutils/core.py", line 219, in publish
    self.apply_transforms()
  File "/usr/lib/python3.8/site-packages/docutils/core.py", line 200, in apply_transforms
    self.document.transformer.apply_transforms()
  File "/usr/lib/python3.8/site-packages/sphinx/transforms/__init__.py", line 80, in apply_transforms
    super().apply_transforms()
  File "/usr/lib/python3.8/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
    transform.apply(**kwargs)
  File "/usr/lib/python3.8/site-packages/sphinx/transforms/__init__.py", line 381, in apply
    self.app.emit('doctree-read', self.document)
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 460, in emit
    return self.events.emit(event, *args, allowed_exceptions=allowed_exceptions)
  File "/usr/lib/python3.8/site-packages/sphinx/events.py", line 105, in emit
    raise ExtensionError(__("Handler %r for event %r threw an exception") %
sphinx.errors.ExtensionError: Handler <function process_posts at 0x7fd4143f2ca0> for event 'doctree-read' threw an exception (exception: 'BuildEnvironment' object has no attribute 'intersphinx_inventory')

Extension error (ablog.post):
Handler <function process_posts at 0x7fd4143f2ca0> for event 'doctree-read' threw an exception (exception: 'BuildEnvironment' object has no attribute 'intersphinx_inventory')

[drammock]

the recommended way to build our own docs is nox -s docs. In fact, nox -s docs does this:

session.run("sphinx-build", "-b=html", "docs/", "docs/_build/html")

which translates to

sphinx-build -b=html docs/ docs/_build/html

which succeeds for me when run from the repository root.

Looking at your traceback, I see this:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 347, in eval_config_file
    exec(code, namespace)
  File "/home/tkloczko/rpmbuild/BUILD/pydata-sphinx-theme-0.10.1/docs/conf.py", line 14, in <module>
    import pydata_sphinx_theme
ModuleNotFoundError: No module named 'pydata_sphinx_theme'

which suggests you don't have the theme properly installed in your environment. I'll close as not reproducible, but if you still have trouble after properly installing the theme, feel free to re-open.

[kloczek]

which succeeds for me when run from the repository root.

This is why I've quoted patch which adds path to pydata_sphinx_theme to sys.path.

[..]

which suggests you don't have the theme properly installed in your environment. I'll close as not reproducible, but if you still have trouble after properly installing the theme, feel free to re-open.

Or it is result of fact that sphinx was not able to locate pydata_sphinx_theme module code .. which fixes patch which I've quoted.
Did you try to experiment with that patch?
Altering sys.path as I've quoted is part of the example/template sphinx conf.py files.

Second thing is that you've been testing with generate html and I've quoted generate roff output (man page).
Many sphinx extension have no proper support to generate anything else than html (despite fact that sphinx supports +15 output documents formats).
Could you please repeat your test with generate man?

[drammock]

sorry, I did not notice that you were trying to build man instead of html. I can confirm that running sphinx-build -b=man docs/ docs/_build/man fails, with:

Extension error (ablog.post):
Handler <function process_posts at 0x7f27030d1ab0> for event 'doctree-read' threw an exception (exception: 'BuildEnvironment' object has no attribute 'intersphinx_inventory')

That can be fixed by adding

    "sphinx.ext.intersphinx",

to the list of extensions in conf.py (around line 23). However, even then the build still fails for me with

Traceback (most recent call last):
  File "/opt/miniforge3/envs/mnedev/lib/python3.10/site-packages/sphinx/cmd/build.py", line 277, in build_main
    app.build(args.force_all, filenames)
  File "/opt/miniforge3/envs/mnedev/lib/python3.10/site-packages/sphinx/application.py", line 349, in build
    self.builder.build_update()
  File "/opt/miniforge3/envs/mnedev/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 298, in build_update
    self.build(['__all__'], to_build)
  File "/opt/miniforge3/envs/mnedev/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 367, in build
    self.write(docnames, list(updated_docnames), method)
  File "/opt/miniforge3/envs/mnedev/lib/python3.10/site-packages/sphinx/util/__init__.py", line 527, in wrapper
    return f(*args, **kwargs)
  File "/opt/miniforge3/envs/mnedev/lib/python3.10/site-packages/sphinx/builders/manpage.py", line 98, in write
    docwriter.write(largetree, destination)
  File "/opt/miniforge3/envs/mnedev/lib/python3.10/site-packages/docutils/writers/__init__.py", line 78, in write
    self.translate()
  File "/opt/miniforge3/envs/mnedev/lib/python3.10/site-packages/sphinx/writers/manpage.py", line 29, in translate
    visitor = self.builder.create_translator(self.document, self.builder)
  File "/opt/miniforge3/envs/mnedev/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 135, in create_translator
    return self.app.registry.create_translator(self, *args)
  File "/opt/miniforge3/envs/mnedev/lib/python3.10/site-packages/sphinx/registry.py", line 359, in create_translator
    setattr(translator, 'visit_' + name, MethodType(visit, translator))
TypeError: first argument must be callable

Note that I do not get the initial error you report (ModuleNotFoundError: No module named 'pydata_sphinx_theme'). I tried it from our repository root (to build our own docs) and I tried it building a totally different project elsewhere on my system. It looks to me like a pretty straightforward "the package is not installed" error; can you explain why you think it needs solving by sys.path.insert? (put another way, why is it not an option to actually install the theme into the environment?)

[kloczek]

can you explain why you think it needs solving by sys.path.insert?

It is very usefull trick which allows:

• guarantee that documentation will be generated out of module code present in source code tree and not (accidentally) out of system installed module (which may be different version)
• generate documentation without have module installed which is especially usefull on packaging because documentation can be generated without use tox or venv (simpler packaging process) or installong module first (which is not possible to do during usuall packaging as such process is performed from non-root account).

[drammock]

Sounds reasonable, though it still won't get us past TypeError: first argument must be callable. I haven't dug into that one yet, @kloczek do you know the right fix there?

@jarrodmillman @choldgraf do you have opinions here? Seems like the vast majority of our users are building HTML, but if supporting buiding man pages is not too difficult (just a couple lines in our conf.py) I see no reason not to do it.

[kloczek]

Man page still is basic/fundamental documentation format on all Unix platforms 😋

[tkloczko@pers-jacek SPECS]$ grep sphinx_build_man python-*spec | wc -l; ls -1 python-*.spec| wc -l
528
1016

So using one unifed rpm macro I was able to generate man pages for more than half of packaged pyton modules.
In each such case I'm adding small patch like for your module

--- a/docs/conf.py
+++ b/docs/conf.py
@@ -63,6 +63,17 @@
 # This allows us to use ::: to denote directives, useful for admonitions
 myst_enable_extensions = ["colon_fence", "substitution"]

+# Man ------------------------------------------------------------------
+man_pages = [
+    (
+        "index",
+        "python-pydata-sphinx-theme",
+        "PyData Sphinx Theme Python Module Documentation",
+        author,
+        3,
+    )
+]
+
 # -- Options for HTML output -------------------------------------------------

 # The theme to use for HTML and HTML Help pages.  See the documentation for

which generates pythonb-<module_name>(3) manpage which ia very usefill when you have commandline completion. Tro access to exact module man page all what you need to do is type $ man python-<tab><tab> and you have list of all python modules documentations. You can grep across that output of search using less (which still is waaaay more usefull than searching on web page)

Back to the topic ..
In such case like failing sphinx basic procedure to encircle extension which is causing crashes is to comment oit one by one of used extensions with ignore warnings about unknown tokens extensing sphinx basix tockens.
Than with identifiew exact module you can add other methods to encircle exact part of the code.

I've not done that because I still have very long list of other packages to revierw/update .. 😞
Temporary I've disabled generate man page for your pydata-sphinx-theme ..

[choldgraf]

I don't know anything about man pages, so can't be of much help here. This theme is definitely focused around html outputs in general. If we can support man without adding much complexity to the theme or maintenance burden, I'm all for it, but I think we are pretty close to our maintenance limit right now already IMO.

[kloczek]

I understand. Thank you 👍

[drammock]

agree with @choldgraf that we (maintainers) are already over-subscribed. If this is something you really want @kloczek you'll need to do the debugging yourself and open a PR; if it doesn't add too much complexity I'd be willing to merge it. But it sounds like you don't even use the theme (?) and you're doing this as a sort of... public service? quixotic pet project? not sure what to call it, so maybe that's a non-starter for you. I'll close on that assumption, but let us know if I'm wrong about that and we can re-open in anticipation of your PR.