setuptools v66.0.0

[regro-cf-autotick-bot]

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

• Dependencies have been updated if changed: see upstream
• Tests have passed
• Updated license if changed and license_file is packaged

Information about this PR:

1. Feel free to push to the bot's branch to update this PR if needed.
2. The bot will almost always only open one PR per version.
3. The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
4. If you want these PRs to be merged automatically, make an issue with @conda-forge-admin,please add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
5. If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Closes: #298

Dependency Analysis

Please note that this analysis is highly experimental. The aim here is to make maintenance easier by inspecting the package's dependencies. Importantly this analysis does not support optional dependencies, please double check those before making changes. If you do not want hinting of this kind ever please add bot: inspection: false to your conda-forge.yml. If you encounter issues with this feature please ping the bot team conda-forge/bot.

Analysis by source code inspection shows a discrepancy between it and the the package's stated requirements in the meta.yaml.

Packages found by source code inspection but not in the meta.yaml:

• platformdirs
• docutils
• pyparsing
• jinja2
• path

_{This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/autotick-bot/actions/runs/3927401495, please use this URL for debugging.}

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[rgommers]

It's possible that this may give some issues because of the removal of non PEP 440 compliant version numbers (see https://setuptools.pypa.io/en/stable/history.html#v66-0-0), but after a day there's no major complaints visible on the setuptools issue tracker yet. So let's give it a go.

[jezdez]

@rgommers FWIW this did break a bunch of things downstream in conda-forge, @jaimergp and @jakirkham know the details, triggered by a conda-build issue of relying on LegacyVersion to parse versions. I think this upgrade should have been bubbled up to more people in the maintainer team to make sure we're ready to handle any issues and not get caught off-guard.

Incidentally, a new conda release has been tagged this week, where this caught my eye. This makes this PR a higher risk change, but probably something you weren't aware of.

So this is an FYI that while you were looking upstream in the packaging issue tracker, it makes sense to also look downstream if people are ready for it :)

[rgommers]

Thanks for flagging this @jezdez.

I think this upgrade should have been bubbled up to more people in the maintainer team to make sure we're ready to handle any issues and not get caught off-guard.

I'm not sure how that could have been done to be honest. setuptools releases very frequently, and every release (major, minor and bugfix) has the potential to break an arbitrary amount of packages. The last massive break was 65.3.1 -> 65.3.2 I believe. setuptools is just hugely unstable, and the only fix is to get rid of it, because we can't run CI for many/all potentially affected packages in conda-forge.

Maybe we can have a policy on this repo that we should wait a week, or 2 weeks, before merging? That's the only thing I can think of.

This makes this PR a higher risk change, but probably something you weren't aware of.

I wasn't indeed. But there's many packages with lots of dependencies that use setuptools, so the risk is pretty much always there.

[jakirkham]

Honestly am not sure what Ralf should do additionally.

Periodically setuptools has broken our tooling. Sometimes this has been hard to anticipate. For a while setuptools released primarily major versions and reading the changelog it was unclear whether it would break something or if the maintainers at the time were being exceedingly cautious.

In terms of the specific issue that occurred with conda-build yesterday ( conda-forge/conda-forge-pinning-feedstock#3973 ), they basically removed something they had been warning about for a while ( pypa/setuptools#2822 (comment) ). While there may be other we could find room to quibble with, this change seems like it was made pretty thoughtfully.

Part of the reason we've developed the infrastructure we have (particularly around conda-forge-ci-setup, repodata hot-fixes, and marking packages broken) is so we can be responsive when these issues occur. In this particular case, we've already hot-fixed existing conda-build packages to not use newer setuptools ( conda-forge/conda-forge-repodata-patches-feedstock#387 ). So additional setuptools updates should be ok.

Getting to the point that I think you are making, how do we handle this going forward. We could...

1. Do downstream testing (as is done elsewhere)
2. Get more involved with upstream packages we depend on
3. Drop setuptools as a conda-build dependency ( Drop dependency on Setuptools conda/conda-build#4443 )

Downstream testing could work, but it seems a bit too much. Users of conda-forge packages should be able to get the latest setuptools if they want. Our build tooling need not constrain that. We can already pin things down as need (which we did here).

Getting more involved upstream could make sense. Though I think the Python world is moving away from everything depending on setuptools to more of a federated model of build tooling sharing mainly specs. So it is unclear whether we want to devote time to setuptools specifically. Thinking more broadly we may want to get more involved in the spec development. However that approaches being tangential to this specific issue (still worth thinking about though).

Really think we should move towards dropping setuptools as a dependency. The fact that build tooling is becoming more federated makes this easier for us to do. Some things that use to only be in setuptools have now been broken out into smaller more focused packages, which may be better suited for our needs. In some cases setuptools has started using these other packages, which makes it easier to identify how such a change can be made and be more confident things will work smoothly. Think we will be happier long term by having a small, focused set of dependencies that meet our needs directly.

[jezdez]

Maybe we can have a policy on this repo that we should wait a week, or 2 weeks, before merging? That's the only thing I can think of.

That seems like a reasonable policy, not sure how easy it is to follow though.

I wasn't indeed. But there's many packages with lots of dependencies that use setuptools, so the risk is pretty much always there.

Something something iceberg below the waterline. I'm glad we're working to move setuptools out of the dependency tree for conda-build.

[jakirkham]

Yeah waiting 2 weeks is tricky. setuptools has historically had very frequent releases. Just this week there were 3 and 2 of them were today.

Honestly the problem the maintainers have there is unenviable. They are carrying a lot of technical debt that they are trying to figure out how to shed.

[jezdez]

Honestly am not sure what Ralf should do additionally.

Periodically setuptools has broken our tooling. Sometimes this has been hard to anticipate. For a while setuptools released primarily major versions and reading the changelog it was unclear whether it would break something or if the maintainers at the time were being exceedingly cautious.

In terms of the specific issue that occurred with conda-build yesterday ( conda-forge/conda-forge-pinning-feedstock#3973 ), they basically removed something they had been warning about for a while ( pypa/setuptools#2822 (comment) ). While there may be other we could find room to quibble with, this change seems like it was made pretty thoughtfully.

Agreed, the removal was quite okay, as @jaimergp mentioned elsewhere, we should probably treat DeprecationWarnings as errors in conda and conda-build to earlier force us to move along.

Part of the reason we've developed the infrastructure we have (particularly around conda-forge-ci-setup, repodata hot-fixes, and marking packages broken) is so we can be responsive when these issues occur. In this particular case, we've already hot-fixed existing conda-build packages to not use newer setuptools ( conda-forge/conda-forge-repodata-patches-feedstock#387 ). So additional setuptools updates should be ok.

That's awesome, and I appreciate that, and yet, I know with my Anaconda hat on, the way this is handled is somewhat different there, less automated, more people focused, which has its upsides and downsides.

Getting to the point that I think you are making, how do we handle this going forward. We could...

1. Do downstream testing (as is done elsewhere)
2. Get more involved with upstream packages we depend on
3. Drop setuptools as a conda-build dependency ( Drop dependency on Setuptools conda/conda-build#4443 )

Downstream testing could work, but it seems a bit too much. Users of conda-forge packages should be able to get the latest setuptools if they want. Our build tooling need not constrain that. We can already pin things down as need (which we did here).

Getting more involved upstream could make sense. Though I think the Python world is moving away from everything depending on setuptools to more of a federated model of build tooling sharing mainly specs. So it is unclear whether we want to devote time to setuptools specifically. Thinking more broadly we may want to get more involved in the spec development. However that approaches being tangential to this specific issue (still worth thinking about though).

Having spent my fair share with setuptools in the past I'm ready to call it what @rgommers already alluded to: if we can find ways for the conda ecosystem to rely on the newer, nimbler "pieces of a puzzle" libraries from PyPA and their related spec work, the better. conda/conda#12245 comes to mind as an interesting recent example, which may have lasting repercussions for the conda ecosystem.

Really think we should move towards dropping setuptools as a dependency. The fact that build tooling is becoming more federated makes this easier for us to do. Some things that use to only be in setuptools have now been broken out into smaller more focused packages, which may be better suited for our needs. In some cases setuptools has started using these other packages, which makes it easier to identify how such a change can be made and be more confident things will work smoothly. Think we will be happier long term by having a small, focused set of dependencies that meet our needs directly.

Thanks for spelling this out very constructively, this is hugely appreciated, I couldn't agree more.

[rgommers]

+1 to all this. Let me make a concrete suggestion. Recently there have been discussions about bootstrapping, after various other pieces of the packaging puzzle broke because of switching backends and circular dependencies. It is now clear that there is one package, flit_core, that supports bootstrapping as a key use case (see https://flit.pypa.io/en/stable/bootstrap.html.). Other foundational packages like pypa/build, wheel, pypa/installer and pip are using it or going to be moving to it. So all the pure Python conda infra packages without special needs beyond stability, and creating sdists/wheels, should probably move to that.

For packages with compiled code (are there any?), best to hold off moving for a little while longer.