Many links to packaging.python.org broken

[takluyver]

I noticed a broken link from the Warehouse help pages to packaging.python.org, and was going to fix it, when I realised every link from Warehouse to packaging.python.org is broken. As are links here on the issue tracker, and links on Discourse posted up to yesterday.

Specifically, a URL like https://packaging.python.org/specifications/core-metadata/ needs an added /en/latest/ to work again. I'm guessing it used to have the 'single version' checkbox selected on RTD, and no longer does ("Use this with caution, only turn it on if you will never have multiple versions of your docs.")

IMO, we should either re-enable that setting ASAP (before people share too many links with the new scheme), or add redirects on RTD so existing links still work.

[bhrutledge]

Thanks for pointing this out. Is there any indication of when this behavior changed? I don't have access to the RTD settings.

I wonder if it has to do with translations, since latest is still the only version listed:

CC @webknjaz @meowmeowmeowcat

[takluyver]

There were some links posted on Discourse at 16:02 UTC yesterday using the old scheme, so I'd guess it was still working then. I don't know of any way to see when exactly it was changed in RTD.

[bhrutledge]

@webknjaz Could this be related to #1023?

[webknjaz]

This is a side-effect of fixing #927 (comment) (TL;DR translations don't work with a "single version mode" enabled). I don't have visibility into the main RTD setup for the English site but I suppose we could collect all the affected documents and manually add redirects for those. This would need to be done by @di or @ewdurbin. I can only provide the list of the URIs to map, it can be scraped via https://webknjaz.github.io/intersphinx-untangled/packaging.python.org/ or via intersphinx directly.

[webknjaz]

This should be the full list:

$ python -m sphinx.ext.intersphinx https://packaging.python.org/en/latest/objects.inv | awk -F': ' '{print$2}' | sed 's/#[^#]\+$//g' | sort -u

contribute/
discussions/
discussions/deploying-python-applications/
discussions/install-requires-vs-requirements/
discussions/pip-vs-easy-install/
discussions/wheel-vs-egg/
genindex/
glossary/
guides/
guides/analyzing-pypi-package-downloads/
guides/creating-and-discovering-plugins/
guides/distributing-packages-using-setuptools/
guides/dropping-older-python-versions/
guides/hosting-your-own-index/
guides/index-mirrors-and-caches/
guides/installing-scientific-packages/
guides/installing-stand-alone-command-line-tools/
guides/installing-using-linux-tools/
guides/installing-using-pip-and-virtual-environments/
guides/making-a-pypi-friendly-readme/
guides/migrating-to-pypi-org/
guides/multi-version-installs/
guides/packaging-binary-extensions/
guides/packaging-namespace-packages/
guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/
guides/single-sourcing-package-version/
guides/supporting-multiple-python-versions/
guides/supporting-windows-using-appveyor/
guides/tool-recommendations/
guides/using-manifest-in/
guides/using-testpypi/
key_projects/
news/
overview/
py-modindex/
search/
specifications/
specifications/binary-distribution-format/
specifications/core-metadata/
specifications/declaring-build-dependencies/
specifications/declaring-project-metadata/
specifications/dependency-specifiers/
specifications/direct-url/
specifications/entry-points/
specifications/platform-compatibility-tags/
specifications/pypirc/
specifications/recording-installed-packages/
specifications/simple-repository-api/
specifications/source-distribution-format/
specifications/version-specifiers/
support/
tutorials/
tutorials/creating-documentation/
tutorials/installing-packages/
tutorials/managing-dependencies/
tutorials/packaging-projects/

[takluyver]

I think that if you add a 'prefix redirect' in RTD with no prefix, any URLs which would 404 would instead be redirected to the equivalent page with /en/latest/ prefixed. I haven't tried this, so take it with a grain of salt, but it might be easier than adding lots of separate redirects.

[webknjaz]

Yeah, I don't remember if it's possible to do for 404 only. Maybe listing just top-level routes would do it:

$ python -m sphinx.ext.intersphinx https://packaging.python.org/en/latest/objects.inv | awk -F': ' '{print$2}' | sed 's/#[^#]\+$//g;s#^\(\w\+/\).*#\1#g' | sort -u

contribute/
discussions/
genindex/
glossary/
guides/
key_projects/
news/
overview/
py-modindex/
search/
specifications/
support/
tutorials/

[webknjaz]

There's also https://sphinx-notfound-page.rtfd.io that can be used as a last resort.

[webknjaz]

Looks like Thomas is right and we should be able to get away with a prefix redirect per https://docs.readthedocs.io/en/stable/user-defined-redirects.html#prefix-redirects.

cc @di @ewdurbin