Pip maintainers: what do you want to know about pip's use?

[nlhkabu]

This issue is specifically for members of @pypa/pip-team

@ei8fdb and I are starting to make a user research plan as part of Phase III of the funded UX work

We have identified 4 "epic" issued we'd like to work on:

1. Carry out research to understand who uses pip (Carry out research to understand who uses pip #8518)
2. Carry out research about pip documentation (Carry out research about pip documentation #8517)
3. Carry out research to understand pip functionality (Carry out research to understand pip functionality #8516)
4. Carry out research to understand pip in the package management ecosystem (Carry out research to understand pip in the package management ecosystem #8515)

In the coming days, we will start to populate these tickets - for example, in the documentation epic, we might list "What do users expect from pip's documentation?" as a question to research.

We already have a fairly good idea of what questions/subjects we need to research, informed by working the @pradyunsg @pfmoore and @uranusjr (and through our previous user research).

However, given that the purpose of this exercise is to inform the pip team, we'd love to hear if you have anything specific you'd like to know about who uses pip and how they use it.

So, if you have a user research question you'd like an answer to, please list it below, and if it's something we feel that we can address, we'll add it to one of the epics.

Thanks!

[pfmoore]

Things I can think of that I'd like to know:

1. What proportion of users are using pip as part of a development workflow (where they do pip install on code of their own that they are developing) versus using pip as a package consumer (just installing other people's packages)? My feeling is that we get disproportionately high levels of feedback from the first category, which means we're (incorrectly) prioritising their needs over that of the second category.
2. What (in general) is the level of experience of the average pip user. What knowledge can we assume, and what must we explain? Do our users expect us to explain basic packaging concepts to them, or are they OK with the idea that there's a "base level" of knowledge that we assume people have?
3. Do the more basic users find the plethora of "advanced" options to pip intimidating, or are they able to ignore them and get what they need?
4. Do you use pip as an individual or as part of a development team?
5. Do you need pip to build projects from source code? Would a "stripped down" version of pip that only installed wheels be sufficient for you? If not, why not? (I'm not proposing that pip drop support for source builds, but there's a lot of pip's code dedicated to source builds, and I'd like to understand whether it's an 80/20 situation - 80% of the code is only needed by 20% of the users...)

As you can see, I have a bias towards wanting to cater for users with less "advanced" needs, because I feel they don't get heard as much as I'd like.

A couple of things for more "advanced" users:

1. How important are security and integrity features like hash checking and reproducible installs? Why? Do you rely solely on pip's features, or do you do anything else? Do you audit the code you install? (I'm basically interested in knowing how much people expect pip to protect them, and how much they have thought about the problem).
2. If you use pip as part of a project development workflow with multiple users, how do you handle code sharing? Do you use a local index server? If not, why not?

[pradyunsg]

Paul stated 4 of my questions already. >.<

Beyond what has already been mentioned, I think it'd be great to get answers to these:

1. what do users think pip is/does? (something like a "tell me, in a sentence or two, what pip does" question in a survey) This is in the direction of understanding what the users' mental model is of "what pip does", and to understand what functionality they view as "main" in pip.

2. which users use which functionality within pip? Epic 1 above is about who uses pip and Epic 3 is about what functionality pip provides -- I want to know the connections between the two. Related to Paul's point 5, but much broader.

3. how/where do users look for information about pip / Python Packaging when (a) they want to accomplish task X and (b) when they get an error message? This is in the direction of better understanding where we should try to put information for the users -- descriptions in the CLI? Documentation? Web searches? StackOverflow? Mailing lists? somewhere else?

4. Do users look that pip's documentation theme and perceive that as "official", since it looks like the CPython documentation? Would they prefer if we change it to something else (say something like https://squidfunk.github.io/mkdocs-material/getting-started/)? [note 1]

5. what are the areas where pip's existing behavior is suboptimal, but users don't report it as an issue / communicate to maintainers about them anymore? eg: places where pip could present a better error message to reduce a complicated discovery step which they might've internalized by now, but other users (or the same users in the future) still get confused by.

6. what issues/roadblocks/speedbreakers do beginners hit? If/how can we smooth these roadblocks? We might want to do this for other user personas/categories, but I feel like that'll get really big in scope very quickly. I don't think any of pip's existing devs/contributors fall into "beginner programmer" category, so we likely don't know enough. :)

[note 1]: This was the original reason we adopted the theme, but I personally really find the navigation capabilities of the theme and available elements/styling to be suboptimal. I really want us to switch over to something with better navigation, design, content elements, search etc for our documentation theme and this would help inform that discussion.

[nlhkabu]

Cross posted to Python discuss - I've invited other packaging project maintainers to also comment here on the assumption that there will be crossover on subjects to research.

If our research can be of use to the wider packaging community, all the better :D

[pradyunsg]

/cc @pypa/pip-committers for their inputs.

[nlhkabu]

If you use pip as part of a project development workflow with multiple users, how do you handle code sharing? Do you use a local index server? If not, why not?

@pfmoore - question: how will knowing this information help you and the team make decisions? Thx

[pfmoore]

Most specifically, we get a fair number of issues raised which can reasonably easily be solved outside of pip by using a custom index server - either something like devpi, or even a small Python script that wraps some logic around the pages served by PyPI.

People are very resistant to using a custom index server, which is somewhat understandable because it's a nuisance to set up. But expecting the pip devs to add more and more complex functionality instead, has its own costs. Basically it's another case where I'd like some concrete evidence to allow us to push back on these types of requests on the basis of keeping pip's scope clear and constrained, rather than only having "we haven't got the resources to do this".

This fits closely with a discussion that the pip devs had not long ago about what is the "scope of pip". At the moment, pip basically adds anything that someone writes a PR for, and then ends up supporting it no matter how many or few users actually care. I strongly believe we need to focus a lot more on "core" pip functionality, and start pushing users to expect to use pip along with other tools, not have everything built into pip. But we can't easily discuss what should be that "core functionality" unless we know what our users really need. (Well, we can - and we have - but the discussion is mainly around what we'd like pip to be, which may not actually align with what other users want 🙁)

[nlhkabu]

Thanks Paul

we get a fair number of issues raised which can reasonably easily be solved outside of pip by using a custom index server

@pfmoore could you point me to some examples so that I can understand better?

[pfmoore]

#8606 is a good example. I think there might also have been others in the past where people had odd network setups that meant they "needed" pip to support their proxy or something else, where actually writing a script or using a proxy like squid to expose a copy of PyPI within their local network would have been a reasonable solution.

[pfmoore]

@nlhkabu Here's another example where a custom index server would be useful, which I just found by accident. This is my comment and this is the overall thread

Basically, users are looking for a way to get dependencies from git, and want pip to be able to find them. But it would be possible for someone to write a custom index server that read the git repo and presented the data using the simple index format, so pip didn't need that capability built in.

[pradyunsg]

Closing this out, since this issue has served the purpose it was meant for. :)