Repository navigation
Switch to GitHub Pages for hosting the documentation #777
Description
Activity
In principle I'm perfectly fine with migrating to GitHub Pages. We don't need the multi-version support that RTD provides, and we also don't need a latest/stable split but can always just deploy the latest dev version of the docs. New features are rare and will anyway be marked with a
.. versionadded::directive.Don't worry about the redirects either, we can quite easily set those up even with URL rewriting that would (for example) drop the
stable/part of each link (I've done it formeson-pythonrecently).For enabling the rest of the interactive documentation for PyWavelets on latest versions, it is then possible to embed a custom Pyodide distribution, which seems to me should be the best approach for reusing the docs. I've previously tried a similar approach, but this new one that I propose should work better, since it would work without the need for nightly WASM wheels.
This is the key part, let's focus on that. I think it's a separate issue, one of the existing ones that deals with nightly wheels probably. I doubt we want to be in the business of hosting a whole distribution, since that doesn't scale very well. I'm still not 100% sure how the recent work by @Carreau et al. gets deployed to get us an up-to-date and working set of packages when loading JupyterLite inside the PyWavelets docs.
I'm also in favor of trying to find a solution that would favor nightly wheels instead of building a whole distribution; It in general better for the ecosystem as it's more flexible and reusable across projects. I'll try to investigate how this ties into JupyterLite.
I think we should also be able to build the wasm32 wheel on RTD as well if we don't want to go upload the wheel to the scientific Python nightly.
I think we should also be able to build the wasm32 wheel on RTD as well if we don't want to go upload the wheel to the scientific Python nightly.
We're already uploading the nightly wheels to https://anaconda.org/scientific-python-nightly-wheels/pywavelets and don't plan to stop, so I think we're fine in that respect.
We're already uploading the nightly wheels to anaconda.org/scientific-python-nightly-wheels/pywavelets and don't plan to stop, so I think we're fine in that respect.
Oh, yeah, I missed that; then we shoudl just pull from there indeed.
Meh, I there is a bug in micorpip that actually mark the wasm32 wheels as incompatible...
Ok, so forcing micropip to accept *_wasm32 works
I think that's because emscripten_3_1_58_wasm32 and pyodide_2024_0_wasm32 are considered different platforms, but are compatible ? Am I wrong ?
Both of them are supposed to be the same platform actually – the rename from the Emscripten version to Pyodide + Year + build number in the platform tag was just to indicate that the built wheels are meant to work with the Pyodide distribution rather than purely wheels that are compiled with Emscripten/for WASM.
It might be a bug,
micropipshould accept both of themI'll try to investigate how this ties into JupyterLite.
I think we should also be able to build the wasm32 wheel on RTD as well if we don't want to go upload the wheel to the scientific Python nightly.
With JupyterLite, the way forward would be that we use https://jupyterlite.readthedocs.io/en/stable/howto/pyodide/packages.html#installing-packages-at-runtime to install the nightly wheels with a new piplite command (and maybe modify
jupyterlite-sphinxto insert that command into a cell automatically). But I think the neater way would be that we do build the wasm32 wheel with RTD (or switch to GitHub Pages like with this proposal), so that we have it installed beforehand (https://jupyterlite.readthedocs.io/en/stable/howto/pyodide/packages.html#bundling-additional-packages-by-default) since the issue to hide a particular cell in a notebook from execution—in this case, the cell at the top of the notebook which would install the nightly wheels—hasn't received much traction: jupyterlite/jupyterlite#508.A custom Pyodide distribution with GitHub Pages would not scale very well, but should be fine for a package like PyWavelets.
Ok, digging into it more, I think the right place to investigate is pyodide-kernel, per jupyterlite-docs this should be the kernel that matters.
worker.ts seem the be the entry point that gets called for both a same-process or in-worker kernels; It itself has options to load packages and url - but hardcoded for now; I think the reasonable things to do is to thread here an extra-option that install packages / url.
The Pyodide kernel itself is created here; which is where data is read from
jupyter-lite.json,
so it should be enough to thread the options from there and update the right interfaces.I'll try to implement that – but so far I'm struggling a bit with having a dev install that consistently reflect my changes. I'm likley doing something wrong with the dev installes of pyodide kernel.
As stated in the issue title, I'd like to propose moving the hosted documentation away from Read the Docs and using GitHub Pages instead for hosting. This was last discussed in #706 (comment) and, probably, a few times elsewhere.
Some of the advantages and reasons for this suggestion are as follows:
.readthedocs.yamlfileSome things to be thought about:
/en/stable/and/en/latest/respectively. Setting redirects for broken links is possible under the Read the Docs admin dashboard, but it's difficult to do the same thing on GitHub Pages.masterbranch and uses a fine-grained PAT to update the rebuilt docs in the gh-pages branch for a PyWavelets/docs repositoryconf.pyfile.