PdmUsageError should include explanation and proposed solution(s)

[palotasb]

Our issue: Inadequate understanding of how PDM works

We're getting started with PDM at my company, switching over from pip freeze >requirements.txt, Poetry, and other solutions. I'm already a big fan of PDM after only using it for a few days because to me it seems like it's the only Python dependency management tool that does things correctly – working with standard pyproject.toml keys, resolving dependencies correctly, defaulting to platform- and Python-version-independent lockfiles, etc. Really great work @frostming and team!

However both while I was converting our project to now use PDM and while my teammates were onboarding to using the new PDM-based tooling, we realized that we don't have a full and correct understand of all of the semantics of various pdm subcommands and how they interact with pyproject.toml, pdm.lock, various dependenciy groups (both in pyproject.toml and pdm.lock!), and the list of actual packages installed, and what happens if any of these are out of sync.

We specifically hit issues #2124 / #2253 and didn't really understand what is causing the [PdmUsageError]: Requested groups not in lockfile errors and what we should do differently to solve the error.

We were adding new dependencies and dependency groups manually to pyproject.toml and running pdm install or pdm install -G :all, but that failed with: [PdmUsageError]: Requested groups not in lockfile. #2124 and #2253 are previously reported issues about this scenario.

Proposed solution: Always offer explanations and workarounds or solutions for usage errors

Usage errors are always errors where PDM knows how to handle the situation (show the error message), but the user doesn't, otherwise they would have not run into the usage error.

As a general guideline I would always include these 2 or 3 things when a PdmUsageError is shown:

• a short explanation of what is happening and why the command invoked by the user is wrong, and/or
• a link to a more thorough explanation of the same (a link to the docs), and
• instructions on what the user should do to resolve the error.

In the specific case of #2124 / #2255:

• The error is that a new dependency group has been added to pyproject.toml, but not to metadata.groups in pdm.lock, and thus pdm install (or pdm install -G :all) won't work.
• The printed explanation should refer to the different concepts of groups within pyproject.toml vs groups within the lockfile and why it won't run when there is a mismatch. (See also our other related enhancement proposal: Better documentation and CLI help for pdm install/lock/update/add semantics #2681.)
• The printed explanation should come with a proposal on what the user should do. E.g., in our case, running pdm lock -G :all and then re-running the pdm install command worked. Another alternative that could be suggested would be running pdm add -G <new-group-name> for each of the new groups. Perhaps editing pdm.lock's metadata would have also worked, we haven't tried. I think it's OK to list all possible solutions or again a link to more details in the docs.

Contribution

Again, happy to contribute, if that helps!

[frostming]

Thank you very much for your detailed feedback, which is very valuable to me.

In fact, I am starting to write a series of articles introducing the internals of PDM. At the same time, I also welcome you to contribute documents. Thank you in advance.

[sr-murthy]

My experience with this error is that it will happen if you add some new dependency groups in your TOML and then run a PDM command, e.g. install or update usually, that reads from a lockfile, usually the default lockfile. If the lockfile hasn't been updated with the new dependency groups then you get that error.

I don't think the error would occur if you update or remove dependency groups, as the information for these would still be in the lockfile, but I guess it seems sensible to update the lockfile for any changes in the TOML.

This issue and and also this partly overlaps with some points I raised in a discussion post, which specifically relates to pdm lock.