Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Command to list all available versions pip finds for a package #7975

Closed
uranusjr opened this issue Apr 4, 2020 · 18 comments
Closed

Command to list all available versions pip finds for a package #7975

uranusjr opened this issue Apr 4, 2020 · 18 comments
Labels
C: cli Command line interface related things (optparse, option grouping etc) type: feature request Request for a new feature UX: functionality research epic Temporary label to link tickets to #8516 UX User experience related

Comments

@uranusjr
Copy link
Member

uranusjr commented Apr 4, 2020

What's the problem this feature will solve?
This is a feature suggestion I got during a user interview conducted by @ei8fdb. Currently pip does not seem to have a way to invoke PackageFinder without also select a “best match”. A common operation for users is however to look at the list of versions, in order to decide how they want to specify the version range.

Describe the solution you'd like
A command in pip to discover and output available versions for a package (or packages). It’d be also useful if the command can also include extra information, e.g. what dists are available for each version, and which version is currently installed, in a table form.

Something like:

$ py -3.8 -m pip discover numpy
I? version	sdist	 wheel
   1.18.2   zip      cp38-cp38m-win_amd64 ...
v  1.18.1   zip  	 cp38-cp38m-win_amd64 ...
(more lines ...)

Each line represents a version. The sdist column shows what format the source dist is in (another common output would be tar.gz), and wheel shows compatible wheels. The dot dot dot at the ends indicates there are more wheels, but incompatible with the host. The I? column shows whether this version is installed. All fields except version can be blank if there’s not a match found.

Similar to install and download, the command would accept index and format options, and filter the output accordingly.

Alternative Solutions
The best solution currently is to manually look at the index directly, but this can be especially cumbersome if the package is not on pypi.org. And even for pypi.org, it is cumbersome to either fire up a browser, or directly use the JSON or simple API.

One other hack the user reported using is to tell pip to install a non-existent version (pip>1000000). pip would output available versions in the error message 😆

Additional context
I didn’t think of any.

@triage-new-issues triage-new-issues bot added the S: needs triage Issues/PRs that need to be triaged label Apr 4, 2020
@uranusjr uranusjr added C: finder PackageFinder and index related code type: feature request Request for a new feature labels Apr 4, 2020
@triage-new-issues triage-new-issues bot removed the S: needs triage Issues/PRs that need to be triaged label Apr 4, 2020
@uranusjr uranusjr added C: cli Command line interface related things (optparse, option grouping etc) and removed C: finder PackageFinder and index related code labels Apr 4, 2020
@McSinyx
Copy link
Contributor

McSinyx commented Apr 4, 2020

I think I can work on this. Is there any option we should consider for this new subcommand?

@uranusjr
Copy link
Member Author

uranusjr commented Apr 4, 2020

I think first we need to reach some concensus among the maintainers this is actually a good idea 🙂

@pfmoore
Copy link
Member

pfmoore commented Apr 4, 2020

+1 from me. This is something I've wanted on a number of occasions.

The layout as described sounds reasonable to me. I can imagine further feature requests asking for extra information in the "I?" column, like "this version would be chosen for installation/upgrade", or "this version is incompatible with what's already installed". Those ideas may be impractical to add, though, and we should be driven by actual use cases, not just by what sounds nice to have (both of those ideas are things I think I'd use in practice, but I have lived without them till now, so they are far from essential).

@uranusjr Do you have any means of involving the user who suggested this, to get their direct input?

@xavfernandez
Copy link
Member

+1 from me also on the general idea.

@uranusjr
Copy link
Member Author

uranusjr commented Apr 4, 2020

Do you have any means of involving the user who suggested this, to get their direct input?

Let me ask Bernard on Zulip, I believe he has ways to contact the interviewee.

@uranusjr
Copy link
Member Author

uranusjr commented Apr 5, 2020

I played around with this and put together a POC. Current output is like this:

> py src\pip find --python-version=3.6 "numpy>=1.15,<1.17"
version sdist wheel
------- ----- --------------------
1.16.6  zip   cp36-cp36m-win_amd64
1.16.5  zip   cp36-cp36m-win_amd64
1.16.4  zip   cp36-cp36m-win_amd64
1.16.3  zip   cp36-cp36m-win_amd64
1.16.2  zip   cp36-cp36m-win_amd64
1.16.1  zip   cp36-cp36m-win_amd64
1.16.0  zip   cp36-cp36m-win_amd64
1.15.4  zip   cp36-none-win_amd64
1.15.3  zip   cp36-none-win_amd64
1.15.2  zip   cp36-none-win_amd64
1.15.1  zip   cp36-none-win_amd64
1.15.0  zip   cp36-none-win_amd64

Code is available at https://github.com/uranusjr/pip/tree/pip-find-versions

@McSinyx
Copy link
Contributor

McSinyx commented Apr 5, 2020

What do you think about adding a column for package name, so that if multiple packages are requested, the output can be easily processed by common CLI utilities?

@gutsytechster
Copy link
Contributor

One other hack the user reported using is to tell pip to install a non-existent version (pip>1000000). pip would output available versions in the error message

I actually do this often. I also think it'd be great to have this feature.
+1 from my side as well(as a pip user)

@McSinyx
Copy link
Contributor

McSinyx commented Apr 6, 2020

@uranusjr, may I continue the feature on top of your branch so you can focus on other works?

@uranusjr
Copy link
Member Author

uranusjr commented Apr 6, 2020

Sure. Regarding the package name, I don’t like the idea since the output would be very repetitive. In the POC I opted to disallow supplying multiple packages instead. An alternative would be to put the name above the table instead. A JSON format can be added for machine-readable output.

One decision I’m pondering on is how to order the versions. The POC puts the latest on top, but I’m thinking maybe it makes more sense to put the latest the the bottom instead, since it’s easy to forget to head the output, and the current ordering requires the user to scroll all the way back for relevant output. OTOH though latest-on-top does make more sense visually (the reverent rows are closer to the header). One alternative would be to introduce a “display x rows at most” flag and set it to a reasonable default, since it is very uncommon for the user to want to see all versions.

@McSinyx
Copy link
Contributor

McSinyx commented Apr 6, 2020

disallow supplying multiple packages

This seems fine to me, until we want to speed up by parallelization (GNU parallel doesnt work since we still need synchronized outputs).

An alternative would be to put the name above the table instead. A JSON format can be added for machine-readable output.

At this point it might be a good idea to provide both human readable and machine-friendly one. JSON IMHO is not something many'd prefer for quick and dirty scraping though.

aptitude's take

$ aptitude versions '~npython3-aiohttp' | cat # for more compact and GitHub-friendly output
Package python3-aiohttp:
p  3.6.2-1+b1 testing 500

Package python3-aiohttp-cors:
p  0.7.0-1 testing 500

Package python3-aiohttp-dbg:
p  3.6.2-1+b1 testing 500

Package python3-aiohttp-jinja2:
p  1.2.0-1 testing 500

Package python3-aiohttp-mako:
p  0.4.0-1 testing 500

Package python3-aiohttp-security:
p  0.4.0-2 testing 500

Package python3-aiohttp-session:
p  2.9.0-2 testing 500

Package python3-aiohttp-socks:
p  0.3.4-1 testing 500

Package python3-aiohttp-wsgi:
p  0.8.2-2 testing 500
$ aptitude versions --group-by=none '~npython3-aiohttp' | cat
p  python3-aiohttp 3.6.2-1+b1 testing 500
p  python3-aiohttp-cors 0.7.0-1 testing 500
p  python3-aiohttp-dbg 3.6.2-1+b1 testing 500
p  python3-aiohttp-jinja2 1.2.0-1 testing 500
p  python3-aiohttp-mako 0.4.0-1 testing 500
p  python3-aiohttp-security 0.4.0-2 testing 500
p  python3-aiohttp-session 2.9.0-2 testing 500
p  python3-aiohttp-socks 0.3.4-1 testing 500
p  python3-aiohttp-wsgi 0.8.2-2 testing 500

latest-on-top does make more sense visually (the reverent rows are closer to the header). One alternative would be to introduce a “display x rows at most” flag and set it to a reasonable default, since it is very uncommon for the user to want to see all versions.

I agreed with this, although it'd be better if we can have more input from the UX research.

@pradyunsg
Copy link
Member

I feel like this makes sense, and we should likely look into aggregating all of our "search, introspection and discovery" commands (like pip search, pip list, pip show, this proposal, proposed pip find and I'm sure there's more) into a better set of not-one-trick-pony commands.

I feel like it's possible to come up with something that's easier to mentally model, and the lower "CLI surface area" will also be easier to maintain as well. Yes, it will require changing more things but I feel that such a fix would improve the overall situation enough to justify the costs.

OTOH, there's no hurry here and seeing what other "gaps" are found during the user research interviews... would be useful. :)

@McSinyx
Copy link
Contributor

McSinyx commented Apr 8, 2020

About the command name, I'm not quite favoring either discover or find for

list all available versions [of] a package

The reason is simply because they seem to imply some discovery, i.e. something more than what the package name can cover (also search, find and discover are synonyms/result of each other). My suggestion as an user of many *nix package managers would be to either

  1. Incorporate the feature as part of pip list, given the similarity of the output format
  2. Use a IMHO more intuitive name like pip versions

I feel like it's possible to come up with something that's easier to mentally model, and the lower "CLI surface area" will also be easier to maintain as well.

IIUC, this also has to do with the growing complexity of pip subcommands. Although admittedly it is convenient (and less to be maintained) to be able to share options between subcommands if we parse them altogether, in longer run it might be beneficial to split them into independent executables, i.e. in the way git does it. I cannot estimate how much work (research and implementation) that would cost though.

@pradyunsg pradyunsg added the UX User experience related label Apr 9, 2020
@sbidoul
Copy link
Member

sbidoul commented Apr 9, 2020

@uranusjr I your example you have a --python-version. Is it the intent to add options to filter on other compatibility tags? Or is the command expected to return only files that are compatible with the current python?

@uranusjr
Copy link
Member Author

@sbidoul My intention was to use it to filter on wheel compatiblity tags and data-requires-python attributes on indexes.

@nlhkabu nlhkabu added the UX: functionality research epic Temporary label to link tickets to #8516 label Jul 28, 2020
@nlhkabu
Copy link
Member

nlhkabu commented Jul 29, 2020

I've added this ticket to the functionality research epic (#8516)
I'll look into how users understand the terms discover and find vs the current search command. My gut feeling is that there will be some confusion.
I'll also research whether this functionality could sit under @pradyunsg's suggestion of creating a pip inspect command (maybe pip inspect --available-versions could work)

@ziebam
Copy link
Contributor

ziebam commented Sep 21, 2021

Hey! Looks like the command was added in GH-8978 and the further improvements are currently tracked in GH-10052.

Should this issue perhaps be closed? 😄

@uranusjr
Copy link
Member Author

Thanks for the reminder 🙂

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 22, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
C: cli Command line interface related things (optparse, option grouping etc) type: feature request Request for a new feature UX: functionality research epic Temporary label to link tickets to #8516 UX User experience related
Projects
None yet
Development

No branches or pull requests

9 participants