Major changes to the documentation

[pradyunsg]

This is basically the supporting issue to #9474, with the idea being to have any broad overarching discussions over here, with code review happening in that PR. :)

[pradyunsg]

So... I started a rewrite of pip's documentation. I'm obviously not done yet, but it's up as a draft very-much-in-progress PR now. I'd like to get feedback on stuff I've done so far, to see if folks do like the general direction this is going. :)

Broadly, the ideas here are:

• Move to Markdown (MyST) for almost all pages.
• Reorganise existing information, to be more... accessible and discoverable.
• Breakup "everything in one page" into multiple "single topic" pages.
  • Rewrite certain sentences, so that there's a consistent style -- mine. :P
• Restructure the entire thing, to organise information in more sensible groups.
  • We're not doing https://documentation.divio.com/ style split-the-4-kinds -- because... I ain't writing new tutorials and how-to guides.

There are 2 "new" pages I want to add (i.e. new content that didn't previously exist), based on documentation feedback from our user research:

• FAQ
• Common Errors

[pradyunsg]

The preview for the draft PR of the rewrite is up at https://pip--9474.org.readthedocs.build/en/9474/. As usual, feedback is welcome. :)

/cc @brainwane @nlhkabu @pypa/pip-committers since I'd really like to hear from them.

[pradyunsg]

Oh yea, and the plan for migration is... to preserve/re-add the old hierarchy as "orphan" pages and add links to the moved content.

And then give search engines ~3-6 months to update to the new locations. We can probably also add RTD redirects at some point, but I'm a little uncomfortable since those aren't surfaced in a config file or whatever. And I don't imagine that being being priority for RTD, since, uhm, it's a team that's stretched really thin already. :)

[pradyunsg]

Dumping a bunch of stackoverflow links, that have either really bad answers or a lot of traffic.

FAQ

https://stackoverflow.com/questions/21055859/what-are-the-risks-of-running-sudo-pip
https://stackoverflow.com/questions/5226311/installing-specific-package-versions-with-pip
https://stackoverflow.com/questions/7225900/how-to-install-packages-using-pip-according-to-the-requirements-txt-file-from-a
https://stackoverflow.com/questions/739993/how-can-i-get-a-list-of-locally-installed-python-modules
https://stackoverflow.com/questions/3220404/why-use-pip-over-easy-install
https://stackoverflow.com/questions/11248073/what-is-the-easiest-way-to-remove-all-packages-installed-by-pip
https://stackoverflow.com/questions/4888027/python-and-pip-list-all-versions-of-a-package-thats-available
https://stackoverflow.com/questions/9510474/removing-pips-cache

Common Errors

https://stackoverflow.com/questions/2817869/error-unable-to-find-vcvarsall-bat
https://stackoverflow.com/questions/8548030/why-does-pip-install-inside-python-raise-a-syntaxerror
https://stackoverflow.com/questions/23708898/pip-is-not-recognized-as-an-internal-or-external-command
https://stackoverflow.com/questions/49768770/not-able-to-install-python-packages-ssl-tlsv1-alert-protocol-version
https://stackoverflow.com/questions/31512422/pip-install-failing-with-oserror-errno-13-permission-denied-on-directory
https://stackoverflow.com/questions/25981703/pip-install-fails-with-connection-error-ssl-certificate-verify-failed-certi
https://stackoverflow.com/questions/35991403/pip-install-unroll-python-setup-py-egg-info-failed-with-error-code-1
https://stackoverflow.com/questions/49748063/pip-install-fails-for-every-package-could-not-find-a-version-that-satisfies
https://stackoverflow.com/questions/52460484/python-3-5-pip-9-attributeerror-nonetype-object-has-no-attribute-bytes

(also, https://pip.pypa.io/en/stable/user_guide/#fixing-conflicting-dependencies moves into one of these)

[pradyunsg]

add RTD redirects at some point, but I'm a little uncomfortable since those aren't surfaced in a config file or whatever

I think there's basically only one class of redirects we'd need to configure here -- reference -> cli stuff:

/reference/pip_cache/ -> /cli/pip_cache/
/reference/pip_check/ -> /cli/pip_check/
/reference/pip_config/ -> /cli/pip_config/
/reference/pip_debug/ -> /cli/pip_debug/
/reference/pip_download/ -> /cli/pip_download/
/reference/pip_freeze/ -> /cli/pip_freeze/
/reference/pip_hash/ -> /cli/pip_hash/
/reference/pip_install/ -> /cli/pip_install/
/reference/pip_list/ -> /cli/pip_list/
/reference/pip_search/ -> /cli/pip_search/
/reference/pip_show/ -> /cli/pip_show/
/reference/pip_uninstall/ -> /cli/pip_uninstall/
/reference/pip_wheel/ -> /cli/pip_wheel/
/reference/pip/ -> /cli/pip/

And redirecting installation to "Quickstart".

And I don't imagine that being being priority for RTD, since, uhm, it's a team that's stretched really thin already. :)

Yup. See readthedocs/readthedocs.org#4221.

[pradyunsg]

Annnd the first PR (#9693) making the necessary non-content changes for this rewrite is filed!

[cjc7373]

Just out of curiosity, why moving from rst to markdown?

[pradyunsg]

Markdown is a significantly more popular markup format than reStructuredText.

It’s likely that potential contributors who are not professional-Python-developers are fairly familiar with Markdown, but haven't used reStructuredText. MyST gives you the best of both worlds – simplicity and familiarity of Markdown with the extensibility power of reST. It's already in use by the Jupyter Book project and is stable enough to write documentation like (most of) https://pradyunsg.me/furo with it.

Also, I think it's much easier to write, and I'm doing a lot of the writing here. :)

[pradyunsg]

I made a GitHub project to track this now, and I'll be updating things there instead of making comments here.

[pradyunsg]

Expand to see current documentation hierarchy (warning: long list)

• Quickstart
• Installation
  • Do I need to install pip?
  • Using Linux Package Managers
  • Using ensurepip
  • Installing with get-pip.py
    • get-pip.py options
  • Upgrading pip
  • Python and OS Compatibility
• User Guide
  • Running pip
  • Installing Packages
  • Basic Authentication Credentials
    • netrc Support
    • Keyring Support
  • Using a Proxy Server
  • Requirements Files
  • Constraints Files
  • Installing from Wheels
  • Uninstalling Packages
  • Listing Packages
  • Searching for Packages
  • Configuration
    • Config file
    • Environment Variables
    • Config Precedence
  • Command Completion
  • Installing from local packages
  • “Only if needed” Recursive Upgrade
  • User Installs
  • Ensuring Repeatability
    • Pinned Version Numbers
    • Hash-checking Mode
    • Installation Bundles
  • Fixing conflicting dependencies
    • Understanding your error message
    • Possible solutions
      • 1. Audit your top level requirements
      • 2. Loosen your top level requirements
      • 3. Loosen the requirements of your dependencies
      • 4. All requirements are loose, but a solution does not exist
    • Getting help
  • Dependency resolution backtracking
    • What is backtracking?
      • How does backtracking work?
    • Why does backtracking occur?
    • What does this behaviour look like?
    • Possible ways to reduce backtracking occurring
      • 1. Allow pip to complete its backtracking
      • 2. Reduce the number of versions pip will try to backtrack through
      • 3. Use constraint files or lockfiles
      • 4. Be more strict on package dependencies during development
    • Getting help
  • Using pip from your program
  • Changes to the pip dependency resolver in 20.3 (2020)
    • Watch out for
    • How to upgrade and migrate
    • Setups to test with special attention
      • Examples to try
    • Tell us about
    • Deprecation timeline
    • Context and followup
• Reference Guide
  • pip
    • Usage
    • Description
      • Logging
        • Console logging
        • File logging
      • --exists-action option
      • Build System Interface
        • Setuptools Injection
        • Build System Output
        • PEP 517 and 518 Support
        • Future Developments
        • Build Options
    • General Options
  • pip install
    • Usage
    • Description
      • Overview
      • Argument Handling
      • Working Out the Name and Version
      • Satisfying Requirements
      • Installation Order
      • Requirements File Format
        • Using Environment Variables
        • Example Requirements File
      • Requirement Specifiers
      • Per-requirement Overrides
      • Pre-release Versions
      • VCS Support
        • Git
        • Mercurial
        • Subversion
        • Bazaar
        • Using Environment Variables
      • Finding Packages
      • SSL Certificate Verification
      • Caching
        • Wheel Cache
      • Hash-Checking Mode
        • Hashes from PyPI
      • Local project installs
        • “Editable” Installs
      • Controlling setup_requires
      • Build System Interface
    • Options
    • Examples
  • pip download
    • Usage
    • Description
      • Overview
    • Options
    • Examples
  • pip uninstall
    • Usage
    • Description
    • Options
    • Examples
  • pip freeze
    • Usage
    • Description
    • Options
    • Examples
  • pip list
    • Usage
    • Description
    • Options
    • Examples
  • pip show
    • Usage
    • Description
    • Options
    • Examples
  • pip search
    • Usage
    • Description
    • Options
    • Examples
  • pip cache
    • Usage
    • Description
    • Options
  • pip check
    • Usage
    • Description
    • Examples
  • pip config
    • Usage
    • Description
    • Options
  • pip wheel
    • Usage
    • Description
      • Build System Interface
        • Customising the build
    • Options
    • Examples
  • pip hash
    • Usage
    • Description
      • Overview
    • Options
    • Example
  • pip debug
    • Usage
    • Description
    • Options
• Development
  • Getting Started
    • Get the source code
    • Development Environment
    • Running pip From Source Tree
    • Running Tests
    • Running Linters
    • Building Documentation
    • What Next?
  • Contributing
    • Pip’s internals
    • Submitting Pull Requests
    • Automated Testing
    • NEWS Entries
      • Contents of a NEWS entry
      • Choosing the type of NEWS entry
    • Updating your branch
    • Becoming a maintainer
  • Continuous Integration
    • Supported interpreters
    • Checks
    • Services
    • Current run tests
      • Developer tasks
      • Actual testing
  • Issue Triage
    • Issue Tracker
      • Labels
      • Automation
    • Triage Issues
      • Requesting information
      • Reproducing issues
      • Reaching resolution
      • Closing issues
    • Common issues
  • Architecture of pip’s internals
    • Broad functionality overview
      • Broad overview of flow
        • Download process
          • PyPI
        • Repository anatomy & directory structure
      • Root and tools
      • src directory
    • Configuration File Handling
      • Overview
      • Configuration
        • Reading from local environment
        • Manipulating a single configuration file
      • kinds
      • Future Refactoring Ideas
    • Finding and choosing files link
      • ... i got lazy ...
    • Command Line Interface
      • Overview
      • Future Refactoring Ideas
    • Options that control the installation process
      • Controlling what gets installed
      • Controlling what gets considered
      • Controlling dependency data
      • Special cases
  • Release process
    • Release Cadence
    • Deprecation Policy
      • Python 2 Support
      • Python Support Policy
    • Feature Flags
      • --use-deprecated
      • --use-feature
    • Release Process
      • Creating a new release
      • Creating a bug-fix release
  • Vendoring Policy
    • Rationale
    • Modifications
    • Automatic Vendoring
    • Debundling
• UX Research & Design
  • How to contribute
    • Participate in UX research
    • Report UX issues
    • Work on UX issues
    • Test new features
  • Next steps
• Changelog
  • 21.1.dev0, unreleased as on (2021-03-20)
    • Process
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 21.0.1 (2021-01-30)
    • Bug Fixes
    • Vendored Libraries
  • 21.0 (2021-01-23)
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 20.3.4 (2021-01-23)
    • Features
    • Bug Fixes
    • Vendored Libraries
  • 20.3.3 (2020-12-15)
    • Bug Fixes
  • 20.3.2 (2020-12-15)
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 20.3.1 (2020-12-03)
    • Deprecations and Removals
  • 20.3 (2020-11-30)
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 20.3b1 (2020-10-31)
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 20.2.4 (2020-10-16)
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Improved Documentation
  • 20.2.3 (2020-09-08)
    • Deprecations and Removals
    • Features
  • 20.2.2 (2020-08-11)
    • Bug Fixes
  • 20.2.1 (2020-08-04)
    • Features
    • Bug Fixes
    • Improved Documentation
  • 20.2 (2020-07-29)
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 20.2b1 (2020-05-21)
    • Bug Fixes
    • Improved Documentation
  • 20.1.1 (2020-05-19)
    • Deprecations and Removals
    • Bug Fixes
  • 20.1 (2020-04-28)
    • Process
    • Features
    • Bug Fixes
  • 20.1b1 (2020-04-21)
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 20.0.2 (2020-01-24)
    • Bug Fixes
    • Vendored Libraries
  • 20.0.1 (2020-01-21)
    • Bug Fixes
  • 20.0 (2020-01-21)
    • Process
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 19.3.1 (2019-10-17)
    • Features
    • Bug Fixes
  • 19.3 (2019-10-14)
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 19.2.3 (2019-08-25)
    • Bug Fixes
  • 19.2.2 (2019-08-11)
    • Bug Fixes
  • 19.2.1 (2019-07-23)
    • Bug Fixes
  • 19.2 (2019-07-22)
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 19.1.1 (2019-05-06)
    • Features
    • Bug Fixes
  • 19.1 (2019-04-23)
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 19.0.3 (2019-02-20)
    • Bug Fixes
  • 19.0.2 (2019-02-09)
    • Bug Fixes
    • Improved Documentation
  • 19.0.1 (2019-01-23)
    • Bug Fixes
  • 19.0 (2019-01-22)
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 18.1 (2018-10-05)
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 18.0 (2018-07-22)
    • Process
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 10.0.1 (2018-04-19)
    • Features
    • Bug Fixes
    • Vendored Libraries
  • 10.0.0 (2018-04-14)
    • Bug Fixes
  • 10.0.0b2 (2018-04-02)
    • Bug Fixes
  • 10.0.0b1 (2018-03-31)
    • Deprecations and Removals
    • Features
    • Bug Fixes
    • Vendored Libraries
    • Improved Documentation
  • 9.0.3 (2018-03-21)
  • 9.0.2 (2018-03-16)
  • 9.0.1 (2016-11-06)
  • 9.0.0 (2016-11-02)
  • 8.1.2 (2016-05-10)
  • 8.1.1 (2016-03-17)
  • 8.1.0 (2016-03-05)
  • 8.0.3 (2016-02-25)
  • 8.0.2 (2016-01-21)
  • 8.0.1 (2016-01-21)
  • 8.0.0 (2016-01-19)
  • 7.1.2 (2015-08-22)
  • 7.1.1 (2015-08-20)
  • 7.1.0 (2015-06-30)
  • 7.0.3 (2015-06-01)
  • 7.0.2 (2015-06-01)
  • 7.0.1 (2015-05-22)
  • 7.0.0 (2015-05-21)
  • 6.1.1 (2015-04-07)
  • 6.1.0 (2015-04-07)
  • 6.0.8 (2015-02-04)
  • 6.0.7 (2015-01-28)
  • 6.0.6 (2015-01-03)
  • 6.0.5 (2015-01-03)
  • 6.0.4 (2015-01-03)
  • 6.0.3 (2014-12-23)
  • 6.0.2 (2014-12-23)
  • 6.0.1 (2014-12-22)
  • 6.0 (2014-12-22)
  • 1.5.6 (2014-05-16)
  • 1.5.5 (2014-05-03)
  • 1.5.4 (2014-02-21)
  • 1.5.3 (2014-02-20)
  • 1.5.2 (2014-01-26)
  • 1.5.1 (2014-01-20)
  • 1.5 (2014-01-01)
  • 1.4.1 (2013-08-07)
  • 1.4 (2013-07-23)
  • 1.3.1 (2013-03-08)
  • 1.3 (2013-03-07)
  • 1.2.1 (2012-09-06)
  • 1.2 (2012-09-01)
  • 1.1 (2012-02-16)
  • 1.0.2 (2011-07-16)
  • 1.0.1 (2011-04-30)
  • 1.0 (2011-04-04)
  • 0.8.3
  • 0.8.2
  • 0.8.1
  • 0.8
  • 0.7.2
  • 0.7.1
  • 0.7
  • 0.6.3
  • 0.6.2
  • 0.6.1
  • 0.6
  • 0.5.1
  • 0.5
  • 0.4
  • 0.3.1
  • 0.3
  • 0.2.1
  • 0.2
  • 0.1.4
  • 0.1.3
  • 0.1.2
  • 0.1.1
  • 0.1
• Code of Conduct
• GitHub