Release 23.1

[pradyunsg]

Filing an issue for this since we're almost at April, and it's nearly time for our next release!

[pradyunsg]

I won't be available for most of the month, outside of a few days at the start -- so someone else from @pypa/pip-committers is gonna have to pick up the mantle of cutting this release. :)

[pradyunsg]

Whomever picks up the RM role for this release, please make sure to assign yourself to this issue.

[pelson]

Is there anything I can do to help (as a non-committer)?

[pradyunsg]

Toward the release process? Not a lot. Toward making the release be better, absolutely!

One thing would be doing a round of vendoring updates. Another would be helping with adding warnings for legacy version/legacy specifiers usage to help with #11715.

https://pip.pypa.io/en/stable/development/release-process/#id5 has the mechanics of the process and the rest is discussions — so there’s little to do on that front.

[pfmoore]

I'm willing to do the RM this time. As before, I'll take a fairly strict "if it's on main, it gets released" approach, so if anyone has anything they want in, get it merged ASAP!

One thought I had - would a mid-week release be better? (To avoid the risk of "everything has been failing all weekend!" issues). I'm flexible, but I honestly don't have a feel for which would be the better option - any thoughts @pypa/pip-committers?

As far as the packaging upgrade goes, I'm assuming that warnings for legacy versions would be in preparation for the upgrade, so if they go in this release, we're heading for a packaging upgrade in 23.2 (July) but if we miss this release with the warnings, we'll need to push the packaging upgrade back an extra 3 months - is that correct @pradyunsg? If so, then getting the warnings sorted is fairly urgent - I'd rather do the release early April rather than at the end of the month, and I don't want to get into a situation of "can we just wait a little while longer, this PR is nearly ready".

I'll go through the changelog in due course, but are there any highlights I should keep an eye out for in this release? The ones I think I recall are:

• Substantial improvements in the resolver, removing all known cases of "never ending" resolves.
• Further work on removing the legacy setup.py install code path.
• The install report format is now stable (and therefore can be considered the canonical way to ask pip "what would you do for this install request?").

[sbidoul]

Hi Paul, thanks for picking this up.

Regarding a release during the week or week-end, both work for me. Just let me know a little bit in advance so I don't pack my agenda too much in the following days so I have some room to help handle any fallout of the setup.py install removal.

Further work on removing the legacy setup.py install code path.

That work is complete with #11874 so you can call that the complete removal of the setup.py install code path.

Related but maybe worth calling out is the fact that --no-binary will only mean one thing, namely "don't download wheels". Which means a wheel will be built always for installation, and the cache of locally built wheels will be used.

[pradyunsg]

is that correct @pradyunsg?

Yes.

If there's anything that's a symptom of my lack of time for pip at the moment, it's that I've written my response 3 times followed by not posting it. Here goes with no additional context. 😅

[pfmoore]

On reflection, I'm going to stick with a weekend release. It suits my schedules, and it's familiar to everyone. My current plan is to aim for 15/16 April, with the following 2 weeks for any possible issues/regressions to be reported, with the aim to have any bugfixes that might be needed done in those 2 weeks and the 23.1 release being closed at the end of April.

Obviously, no plan survives contact with the enemy, but we'll see how this goes.

So anyone wanting to get a feature into 23.1 should aim to get it merged in the next 2 weeks. And if you're going to be close to the deadline, link to the PR and underlying issue here, so I'm aware.

[pfmoore]

I just took a quick look at vendoring updates. It looks like the pygments patch needs updating. Can anyone explain to me how to fix up a vendoring patch, starting from the current main branch? If I do nox -s vendor -- --upgrade pygments I get a bunch of changed files, and I can see the lines that need fixing, but I don't know how to create a patch stanza in the right format to add to the existing pygments.patch file.

I don't mind if someone else ends up picking up the vendoring updates, but whoever does it I think we should probably document the steps needed here, as it's currently a bottleneck if we're relying on one or two people knowing how to do this, and it's uncommon enough that expecting people to remember from last time is probably naïve at best 🙂

[pradyunsg]

Do you wanna do an upgrade skipping pygments? I'll try to find time to do that upgrade, and document the process -- it's the only part of the vendoring process that I've not had time to polish up: pradyunsg/vendoring#4

[pfmoore]

Sure, I can do that. Thanks.

[pfmoore]

Hmm, looks like tenacity and pkg_resources patches also need updating 🙁 I had a go at fixing the pkg_resources one, but somehow the patch file got committed in a setuptools vendoring, which might be right, but I can't even tell from reading the code how everything from setuptools apart from pkg_resources gets removed, or how the patches even get applied (there's nothing in the noxfile, and the patch directory isn't passed to vendoring. So unless vendoring has hard coded values for the patch directory and the handling of setuptools (and I couldn't find any such code) I'm baffled.

I'm not confident that I'm doing the right thing(s) here, so I've skipped everything that isn't clean using

nox -s vendoring -- --upgrade-all --skip packaging --skip pygments --skip tenacity --skip setuptools

Resulting PR is #11916

Edit: I got the patches for the 3 libraries that wouldn't apply cleanly sorted, so that's all of the vendoring, I believe.

[pradyunsg]

Vendoring is configured through pyproject.toml entirely. You can find the bits related to setuptools trimming there.

[pradyunsg]

Thanks for handling all of the vendoring stuff Paul! ^>^

[kasium]

Any change to also update the packaging vendored Version finally? See #9613

[pradyunsg]

See #11715 for that -- that and linked issues/PR will have the relevant context.

[pfmoore]

OK, all relevant PRs/issues on the milestone are now complete (or in the case of "Truststore by default", moved to 23.2). I'm going to start the release process now.

[pfmoore]

Release is now complete!

[pfmoore]

I'll be doing a 23.1.1 which reverts the upgrade of pkg_resources, because of #11975. As the only other changes on the main branch at the moment are documentation updates, I'll probably cut 23.1.1 from main to avoid needing to cherrypick. So please exercise caution in merging PRs until the bugfix is released. It's not the end of the world - I can do a cherrypick if needed - I'm just trying to keep things simple.

I'll do 23.1.1 in the next day or so, depending on whether we get any more issues reported.

[pfmoore]

Just started preparing release 23.1.1.

[pfmoore]

... and 23.1.1 has been released!

[pfmoore]

Pip 23.1.2 has now been released. This re-upgrades setuptools with a fix to avoid the pkg_resources deprecation warning introduced in the latest version.

[pfmoore]

OK, I'm going to say that 23.1 is now done. The setuptools upgrade has unblocked the CPython changes to remove imp for 3.12, so I think we're good at this point.