Skip to content

Conversation

@steppi
Copy link
Collaborator

@steppi steppi commented Mar 17, 2024

Closes #146

This PR adds return {"parallel_read_safe": True} to setup in jupyterlite_sphinx.py, marking jupyterlite_sphinx as parallel read safe in the metadata.

Without this, Sphinx forces a serial read

WARNING: the jupyterlite_sphinx extension does not declare if it is safe for parallel reading,
assuming it isn't - please ask the extension author to check and make it explicit
WARNING: doing serial read

I've determined that the extension is parallel read safe. I think the only point of contention is the use of self.env.temp_data seen below, to avoid regenerating notebooks if a docstring is processed multiple times.

if "generated_notebooks" not in self.env.temp_data:
self.env.temp_data["generated_notebooks"] = {}
directive_key = f"{self.env.docname}-{self.lineno}"
notebook_unique_name = self.env.temp_data["generated_notebooks"].get(
directive_key
)

and later on

if notebook_unique_name is None:
nb = examples_to_notebook(self.content, warning_text=warning_text)

Since the keys in the dictionary are unique per appearance of the directive, and the only operations are setting for a key if the key doesn't exist, or checking if the key is in the dictionary, parallel reads should have no impact. I've tested this building SciPy's documentation with parallel reading, and found no issues.

@steppi steppi added the enhancement New feature or request label Mar 17, 2024
@steppi steppi changed the title Add metadata for parallel_read_safe = True ENH: Add metadata for parallel_read_safe = True Mar 17, 2024
@Carreau Carreau merged commit f66e6c3 into jupyterlite:main Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adding metadata for parallel_read_safe

2 participants