pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf

[mkoeppe]

Similar to #29411 + #29950, which made sagelib a pip-installable distribution (which is then installed by the sage-the-distribution script package sagelib), we change the build/installation scheme of the sage documentation.

We create pip-installable packages sagemath-doc-html and sagemath-doc-pdf. By providing wheels on pypi, users can get access to an offline copy of the documentation without having to build it themselves.

We create a package sagemath-doc-inventory, which is in charge of building and installing the inventory files.
It is a build dependency of sagemath-doc-html and sagemath-doc-pdf.

All of these packages depend on sagemath-doc-src. It has a build-system dependency on sagelib. An sdist of it consists of SAGE_SRC/doc. A wheel of sagemath-doc-src (and thus an installation) contains a copy of the sources (including the auto-generated rst files) in share/doc/sage/src/ and the doctrees in share/doc/sage/doctrees.
The incremental build features of the docbuild will only be active when the package sagemath-doc-src is installed directly using setup.py install, setup.py develop or pip install --editable. In these cases, the doctrees (including environment pickes) are kept in the source directory.
pip wheel will always make a from-scratch build of the documentation, eliminating incremental build errors.

The script sage-grepdoc will be provided by sagemath-doc-html.

Follow-up tickets:

• When installed using pip, the packages provide nbextensions that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see Meta-ticket: Use system Jupyter notebook / JupyterLab #30306 (3)).
  • https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Distributing%20Jupyter%20Extensions%20as%20Python%20Packages.html
• Or alternatively as a JupyterLab extension
  • https://jupyterlab.readthedocs.io/en/stable/extension/extension_dev.html

See also:

• Split sage_setup.docbuild out to a separate distribution sage_docbuild #30010: Split sage_setup.docbuild out to a separate package
• build/make/Makefile.in: Rename make targets SPKG-clean to SPKG-uninstall #29097: build/make/Makefile.in: Rename make targets SPKG-clean to SPKG-uninstall
• add intersphinx mapping for SciPy #29231: Add intersphinx mapping for SciPy -- with discussion about docbuild and inventory files

Depends on #33852

CC: @isuruf @kiwifb @antonio-rojas @timokau @nbruin @dimpase @jhpalmieri @mwageringel @zlscherr

Component: build

Keywords: sd111

Issue created by migration from https://trac.sagemath.org/ticket/29868

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,3 +1,5 @@
 Similar to #29411, which makes sagelib a script package, we make the sage docbuild a script package.
 
 
+See also: #30010 - Split `sage_setup.docbuild` out to a separate package
+

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,5 +1,8 @@
-Similar to #29411, which makes sagelib a script package, we make the sage docbuild a script package.
+Similar to #29411 + #29950, which made `sagelib` a pip-installable distribution (which is then installed by the sage-the-distribution script package `sagelib`), we change the installation scheme of the sage documentation.
 
+We create pip-installable packages `sage-doc-html` and `sage-doc-pdf`. 
+
+When installed using pip, they provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).
 
 See also: #30010 - Split `sage_setup.docbuild` out to a separate package
 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -4,5 +4,7 @@
 
 When installed using pip, they provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).
 
+Also, the script `sage-grepdoc` will be provided by `sage-doc-html`.
+
 See also: #30010 - Split `sage_setup.docbuild` out to a separate package
 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -6,5 +6,7 @@
 
 Also, the script `sage-grepdoc` will be provided by `sage-doc-html`.
 
-See also: #30010 - Split `sage_setup.docbuild` out to a separate package
+See also: 
+- #30010 - Split `sage_setup.docbuild` out to a separate package
+- #29097 - `build/make/Makefile.in`: Rename make targets `SPKG-clean` to `SPKG-uninstall`
 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,8 +1,8 @@
 Similar to #29411 + #29950, which made `sagelib` a pip-installable distribution (which is then installed by the sage-the-distribution script package `sagelib`), we change the installation scheme of the sage documentation.
 
-We create pip-installable packages `sage-doc-html` and `sage-doc-pdf`. 
+We create pip-installable packages `sage-doc-html` and `sage-doc-pdf`. By providing wheels on pypi, users can get access to an offline copy of the documentation without having to build it themselves.
 
-When installed using pip, they provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).
+When installed using pip, the packages provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).
 
 Also, the script `sage-grepdoc` will be provided by `sage-doc-html`.
 

[mkoeppe]

Changed keywords from none to sd111

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,10 +1,10 @@
 Similar to #29411 + #29950, which made `sagelib` a pip-installable distribution (which is then installed by the sage-the-distribution script package `sagelib`), we change the installation scheme of the sage documentation.
 
-We create pip-installable packages `sage-doc-html` and `sage-doc-pdf`. By providing wheels on pypi, users can get access to an offline copy of the documentation without having to build it themselves.
+We create pip-installable packages `sagemath-doc-html` and `sagemath-doc-pdf`. By providing wheels on pypi, users can get access to an offline copy of the documentation without having to build it themselves.
 
 When installed using pip, the packages provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).
 
-Also, the script `sage-grepdoc` will be provided by `sage-doc-html`.
+Also, the script `sage-grepdoc` will be provided by `sagemath-doc-html`.
 
 See also: 
 - #30010 - Split `sage_setup.docbuild` out to a separate package

[mkoeppe]

Description changed:

--- 
+++ 
@@ -9,4 +9,5 @@
 See also: 
 - #30010 - Split `sage_setup.docbuild` out to a separate package
 - #29097 - `build/make/Makefile.in`: Rename make targets `SPKG-clean` to `SPKG-uninstall`
+- #29231 (add intersphinx mapping for SciPy) - with discussion about docbuild and inventory files
 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,10 +1,18 @@
-Similar to #29411 + #29950, which made `sagelib` a pip-installable distribution (which is then installed by the sage-the-distribution script package `sagelib`), we change the installation scheme of the sage documentation.
+Similar to #29411 + #29950, which made `sagelib` a pip-installable distribution (which is then installed by the sage-the-distribution script package `sagelib`), we change the build/installation scheme of the sage documentation.
 
 We create pip-installable packages `sagemath-doc-html` and `sagemath-doc-pdf`. By providing wheels on pypi, users can get access to an offline copy of the documentation without having to build it themselves.
 
-When installed using pip, the packages provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).
+We create a package `sagemath-doc-inventory`, which is in charge of building and installing the inventory files.
+It is a build dependency of `sagemath-doc-html` and `sagemath-doc-pdf`.
 
-Also, the script `sage-grepdoc` will be provided by `sagemath-doc-html`.
+We no longer install `doctrees` in `$SAGE_LOCAL/share/doc/sage/`.
+The incremental build features of the docbuild will only be active when the package is installed directly using `setup.py install`, `setup.py develop` or `pip install --editable`.  In these cases, the doctrees (including environment pickes) are kept in the source directory.
+`pip wheel` will always make a from-scratch build of the documentation, eliminating incremental build errors.
+
+The script `sage-grepdoc` will be provided by `sagemath-doc-html`.
+
+Follow-up tickets:
+- When installed using pip, the packages provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).
 
 See also: 
 - #30010 - Split `sage_setup.docbuild` out to a separate package

[mkoeppe]

Description changed:

--- 
+++ 
@@ -5,8 +5,8 @@
 We create a package `sagemath-doc-inventory`, which is in charge of building and installing the inventory files.
 It is a build dependency of `sagemath-doc-html` and `sagemath-doc-pdf`.
 
-We no longer install `doctrees` in `$SAGE_LOCAL/share/doc/sage/`.
-The incremental build features of the docbuild will only be active when the package is installed directly using `setup.py install`, `setup.py develop` or `pip install --editable`.  In these cases, the doctrees (including environment pickes) are kept in the source directory.
+All of these packages depend on `sagemath-doc-src`. It has a build-system dependency on sagelib. An `sdist` of it consists of `SAGE_SRC/doc`. A `wheel` of `sagemath-doc-src` (and thus an installation) contains a copy of the sources (including the auto-generated rst files) in `share/doc/sage/src/` and the doctrees in `share/doc/sage/doctrees`.
+The incremental build features of the docbuild will only be active when the package `sagemath-doc-src` is installed directly using `setup.py install`, `setup.py develop` or `pip install --editable`.  In these cases, the doctrees (including environment pickes) are kept in the source directory.
 `pip wheel` will always make a from-scratch build of the documentation, eliminating incremental build errors.
 
 The script `sage-grepdoc` will be provided by `sagemath-doc-html`.

[mkoeppe]

Dependencies: #31356

[mkoeppe]

comment:14

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[mkoeppe]

comment:15

Input and help from docbuild experts is very welcome.

[mkoeppe]

comment:16

Some questions:

• Who generates files such as src/doc/en/reference/monoids/sage/monoids/free_monoid.rst, and should they be part of what sagemath-doc-src installs (and thus ships with the wheel)?

• Is there a way to only generate the doctrees but not build HTML or PDF?

• Do the HTML and PDF builds need to be able to run the Sage library, or can they build these formats if given only the doctrees?

[kiwifb]

comment:17

Replying to @mkoeppe:

Some questions:

• Is there a way to only generate the doctrees but not build HTML or PDF?

I do believe there is a sphinx option for doing just that.

• Do the HTML and PDF builds need to be able to run the Sage library, or can they build these formats if given only the doctrees?

As far as I know they need to be able to run the sage library. In particular all graphs/plots are generated during the documentation build.