Issue in 3.1.1: Handler <function run_autoapi at 0x7c590c5005e0> for event 'builder-inited' threw an exception (exception: [Errno 2] No such file or directory

[castlez]

This is new as of today, i have a build from yesterday that built the docs ok. Ive mentioned this in #441 but ill reiterate here for completeness.

Just another data point here, i get the same thing if i run in a venv but not if i use system python (granted i still get an error, but i think its unrelated to this issue)

versions:

➜  python --version
Python 3.11.9
---------------
➜  pip freeze
...
Sphinx==7.3.7
sphinx-autoapi==3.1.1
sphinx-rtd-theme==2.0.0
sphinxcontrib-applehelp==1.0.8
sphinxcontrib-devhelp==1.0.6
sphinxcontrib-htmlhelp==2.0.5
sphinxcontrib-jquery==4.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.7
sphinxcontrib-serializinghtml==1.1.10
...

Here's installing deps:

➜ pip install sphinx
➜ pip install sphinx_rtd_theme
➜ pip install sphinx-autoapi

Here's the commands

➜  tree .
.
├── build
│   └── _static
│       ├── jquery.js
│       └── _sphinx_javascript_frameworks_compat.js
├── code_src
│   ├── dummy.py
│   ├── __init__.py
│   └── __pycache__
│       ├── dummy.cpython-311.pyc
│       └── __init__.cpython-311.pyc
├── docs
│   ├── make.bat
│   ├── Makefile
│   └── source
│       ├── conf.py
│       └── index.rst
├── main.py
└── requirements.txt
➜  cd docs/source
➜  sphinx-build -b html . ../../build

and lastly here's the conf.py

# Configuration file for the Sphinx documentation builder.
#
# For the full list of built-in configuration values, see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
templates_path = ["_templates"]
source_suffix = ".rst"
master_doc = "index"
suppress_warnings = ["autoapi"]


project = 'GitRequester'
copyright = '2024, Castle'
author = 'Castle'
release = '1.0.0'

# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
    'autoapi.extension',
    'sphinx.ext.viewcode'
]

autoapi_type = 'python'
autoapi_dirs = ['../../code_src']
autoapi_file_pattern = "*.py"
autoapi_options = ['members', 'private-members', 'show-inheritance',
                   'show-module-summary', 'special-members', 'imported-members', ]

exclude_patterns = ['build', 'Thumbs.db', '.DS_Store']
pygments_style = "sphinx"

# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
#
html_theme = 'sphinx_rtd_theme'

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']

Here's the output, this was not happening yesterday before this went in eecb6e8 (I am not 100% confident this change made it happen, i just see a correlation between this going in and this issue)

➜  sphinx-build -b html . ../../build
Running Sphinx v7.3.7
WARNING: html_static_path entry '_static' does not exist
[AutoAPI] Reading files... [100%] /home/castle/src/sphinx_poc/code_src/__init__.py
[AutoAPI] Mapping Data... [100%] /home/castle/src/sphinx_poc/code_src/__init__.py
[AutoAPI] Rendering Data...

Extension error (autoapi.extension):
Handler <function run_autoapi at 0x7cced9cfc5e0> for event 'builder-inited' threw an exception (exception: [Errno 2] No such file or directory: '/home/castle/src/sphinx_poc/docs/source/autoapi/index.rst')

For what its worth, i have been using sphinx-autoapi for a while now, and only recently started seeing this. Hard to pinpoint because i dont redeploy docs everyday, but definitely in the last month this came up.

Not sure why its looking in docs/source/autoapi/index.rst thats never where the index has been

[castlez]

Just confirmed that pinning sphinx-autoapi to version 3.1.0 solves the issue, meaning it was indeed something (somehow) in the 3.1.1 release. Looking at the code, i dont know enough to be able to pin point it, but im more confident its in there somewhere now. Gonna update the issue title to reflect that

[alexfikl]

This just started happening to me too recently. It seems like the issue is that build_finished tries to delete the autoapi_root here

sphinx-autoapi/autoapi/extension.py

Lines 149 to 159 in 8001fbd

	def build_finished(app, exception):
	if not app.config.autoapi_keep_files and app.config.autoapi_generate_api_docs:
	normalized_root = os.path.normpath(
	os.path.join(app.srcdir, app.config.autoapi_root)
	)
	if app.verbosity > 1:
	LOGGER.info(
	colorize("bold", "[AutoAPI] ")
	+ colorize("darkgreen", "Cleaning generated .rst files")
	)
	shutil.rmtree(normalized_root)

but it fails, since the folder does not exist on my system, not sure why. I've worked around it by adding this to conf.py

autoapi_keep_files = True

but it would be nice to know why this started happening and what's wrong in my config :\

[sachahu1]

I just ran into the same issue, here's what I found:

I believe the autoapi_options that is causing all this fuss is "undoc-members". It is present in the default value but if you remove it sphinx-autoapi doesn't produce rst files for anything "undocumented".

So what counts as undocumented?

The short answer is: every module needs a docstring.

1. If you have a nicely documented function in file.py but no "module" docstring, nothing from that file will show up.
2. If file.py has a module docstring but lives inside a package whose __init__.py doesn't have a module docstring, nothing will show up.

So for something to be documented by autoapi, all of his parents need to be documented and they must all have module docstrings.
So in the project below, for anything from schema_one.py to show up, you will need to have docstrings as follows:

doc_test_project/
├── another_module.py
├── cli.py
├── __init__.py <-- Module docstring
└── schemas
    ├── __init__.py <-- Module docstring
    └── schema_one.py <-- Document function + module

The solution

Either add module docstrings everywhere (or find some other solution to force autoapi to document these files) or add the "undoc-members" option back into the autoapi_options list

Actual permanent fix

From a code point of view, I'm still trying to trace down the origin of the problem but I think the problem stems from here:

sphinx-autoapi/autoapi/_mapper.py

Lines 337 to 343 in 8001fbd

	for _, obj in status_iterator(
	self.objects_to_render.items(),
	colorize("bold", "[AutoAPI] ") + "Rendering Data... ",
	length=len(self.objects_to_render),
	verbosity=1,
	stringify_func=(lambda x: x[0]),
	):

I think due to the parameters, self.objects_to_render.items() ends up being empty.

This seems to be caused by

sphinx-autoapi/autoapi/_objects.py

Lines 222 to 242 in eecb6e8

	def _should_skip(self) -> bool:
	skip_undoc_member = self.is_undoc_member and "undoc-members" not in self.options
	skip_private_member = (
	self.is_private_member and "private-members" not in self.options
	)
	skip_special_member = (
	self.is_special_member and "special-members" not in self.options
	)
	skip_imported_member = self.imported and "imported-members" not in self.options
	skip_inherited_member = (
	self.inherited and "inherited-members" not in self.options
	)
	return (
	self.obj.get("hide", False)
	or skip_undoc_member
	or skip_private_member
	or skip_special_member
	or skip_imported_member
	or skip_inherited_member
	)

returning True for everything.

I would expect even modules without a top-level module docstring should still be documented by autoapi.

@alexfikl (Tagged you since it might answer your question)

Let me know if that helps!

[alexfikl]

@sachahu1 Can confirm that adding back undoc-members to autoapi_options fixes this for me. Thank you for the explanation!

[AWhetter]

I think that it's legitimate for a module with no docstring to not get rendered unless undoc-members is set. undoc-members can be a useful control for what gets documented and what doesn't, and this functionality should extend to modules as well.
It's not great that this behaviour changed in a minor release though. It probably warranted a major release, so apologies for the disruption caused.
As for this error, it seems to come up when no module is documented and index injection occurs. AutoAPI should raise a more friendly error or warning in this case.

[taldcroft]

I think that it's legitimate for a module with no docstring to not get rendered unless undoc-members is set. undoc-members can be a useful control for what gets documented and what doesn't, and this functionality should extend to modules as well.

While you have a point if being strictly consistent about the interpretation of undoc-members at different levels, in practice this is problematic for typical users. A key reason I would like to use this sphinx-autoapi is to document an existing package without making significant changes to the code itself. In our org we have several dozen large packages where I'm considering using sphinx-autoapi, but making wholesale code changes is not an option.

• There are many undoc-members that only clutter the doc pages, so it helps to exclude them.
• Most of the modules do not have a docstring.

Could you add an option to special-case modules that do not have a docstring?

As an aside, the update in 5222df7 did not quite do the job of helping me (as a user) figure out what was happening. It never occurred to me that modules would be excluded because of being an undoc-member. The sphinx build process then ended badly:

writing output... [ 20%] 
archived_topics


Theme error:
An error happened in rendering the page archived_topics.
Reason: KeyError('technical/api/index')
make: *** [html] Error 2

I was only able to understand what was happening by googling and finding this issue.