Enable linking multiple Python formulae #107517

carlocab · 2022-08-07T06:24:16Z

python@3.9: use altinstall target
python@3.10: link into HOMEBREW_PREFIX
python@3.9: cleanup
python@3.10: cleanup
Update Python aliases
versioned_keg_only_allowlist: add Python 3.10

The primary change (see first commit) uses the altinstall target in the Python Makefile to make sure that Python3.9 only installs files that contain Python's major and minor version. That is, python@3.9 will now install only python3.9, python3.9-config, etc., instead of python3 and python3-config.

This will allow users to have both python@3.9 and python@3.10 linked into HOMEBREW_PREFIX simultaneously. Users who wish to call python3.9 as python3 can add libexec/"bin" to their PATH.

Going forward, I believe this will help simplify migrating to newer Python versions, since there will be no need to make the newer version keg-only while most dependent formulae still depend on the older version of Python.

I've made a lot of changes here. For ease of review, I placed substantive changes in the first two commits. The third and fourth commits are refactoring, and the ~~last two~~ fifth and sixth update non-formula files.

I have not tested this on Linux. This may require a few more adjustments there.

See discussion at Homebrew/discussions#2776 and documentation at https://github.com/python/cpython#installing-multiple-versions.

Closes #87075.

Have you followed the guidelines for contributing?
Have you ensured that your commits follow the commit style guide?
Have you checked that there aren't other open pull requests for the same formula update/change?
Have you built your formula locally with brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
Does your build pass brew audit --strict <formula> (after doing brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

Formula/python@3.10.rb

Formula/python@3.9.rb

Formula/python@3.10.rb

Formula/python@3.9.rb

Rylan12

Doesn't this mean that if you run brew install python@3.9 (no alias so no auto upgrade to python@3.10) and then have a script that relies on /opt/homebrew/bin/python3 it will no longer work unless you manually install python@3.10?

carlocab · 2022-08-07T16:00:29Z

Doesn't this mean that if you run brew install python@3.9 (no alias so no auto upgrade to python@3.10) and then have a script that relies on /opt/homebrew/bin/python3 it will no longer work unless you manually install python@3.10?

Yes. But the same thing happens whenever we make python@3.9 keg-only and link python@3.10, so the way we've been handling this so far has the same problem.

carlocab · 2022-08-08T01:42:38Z

Formula/python@3.9.rb

+      If you do not need a specific version of Python, and always want Homebrew's `python3` in your PATH:
+        brew install python3


@Rylan12 this should help a bit with users who really just want python3 (of whatever version) but did brew install python@3.9.

It won't make it go away, since not everyone reads the caveats, but it should help a bit.

Formula/python@3.10.rb

Co-authored-by: XuehaiPan <XuehaiPan@pku.edu.cn>

carlocab · 2022-08-11T10:12:07Z

This is ready for review.

Some dependents will still fail, but all failures should have fixes in progress. See #107517 (comment).

I hope to be able to merge this once CI completes.

MikeMcQuaid

Makes sense to me. One small question.

MikeMcQuaid · 2022-08-11T10:23:21Z

Formula/python@3.10.rb

+  link_overwrite "Frameworks/Python.framework/Headers"
+  link_overwrite "Frameworks/Python.framework/Python"
+  link_overwrite "Frameworks/Python.framework/Resources"
+  link_overwrite "Frameworks/Python.framework/Versions/Current"


Does this mean whichever version is linked last will be Current?

I contemplated that, but found it too confusing, so decided against it.

In this PR, only the newest Python in Homebrew/core (i.e. python@3.10) will have these symlinks.

I would like to find a way to do it so that the newest Python you have installed will have these symlinks, so that the behaviour is consistent and predictable, but im punting on that for now.

I contemplated that, but found it too confusing, so decided against it.

Agreed, I think it's better this way.

In this PR, only the newest Python in Homebrew/core (i.e. python@3.10) will have these symlinks.

How do we ensure these get updated/migrated on a new Python version? Audit? Could do something like fail if the alias doesn't point to python.

How do we ensure these get updated/migrated on a new Python version? Audit? Could do something like fail if the alias doesn't point to python.

Yup, we should add an audit for this. Will look at one soon.

@carlocab Sounds great, doesn't need to block this PR 👍🏻

This is a backport of Homebrew#107517 to `python@3.8`.

BrewTestBot · 2022-08-12T03:50:32Z

@carlocab has triggered a merge.

This is a backport of #107517 to `python@3.8`.

This is a backport of Homebrew#107517 to `python@3.7`.

Fixes a CI failure seen at Homebrew#107517.

This is a backport of #107517 to `python@3.7`.

BrewTestBot added the automerge-skip `brew pr-automerge` will skip this pull request label Aug 7, 2022

carlocab mentioned this pull request Aug 7, 2022

python@3.10: migrate aliases #87075

Closed

6 tasks

carlocab requested a review from a team August 7, 2022 06:26

carlocab force-pushed the link-all-pythons branch 6 times, most recently from a56c57e to 6598029 Compare August 7, 2022 09:22

carlocab commented Aug 7, 2022

View reviewed changes

Formula/python@3.10.rb Outdated Show resolved Hide resolved

carlocab force-pushed the link-all-pythons branch 2 times, most recently from b19b708 to 5a5f61a Compare August 7, 2022 09:36

carlocab commented Aug 7, 2022

View reviewed changes

Formula/python@3.10.rb Outdated Show resolved Hide resolved

Formula/python@3.9.rb Outdated Show resolved Hide resolved

carlocab commented Aug 7, 2022

View reviewed changes

Formula/python@3.10.rb Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

carlocab commented Aug 7, 2022

View reviewed changes

Formula/python@3.9.rb Show resolved Hide resolved

carlocab force-pushed the link-all-pythons branch from 5a5f61a to e0a4730 Compare August 7, 2022 13:12

carlocab added long build Set a long timeout for formula testing CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. CI-linux-self-hosted Build on Linux self-hosted runner labels Aug 7, 2022

Rylan12 reviewed Aug 7, 2022

View reviewed changes

carlocab force-pushed the link-all-pythons branch from e0a4730 to 7472b2d Compare August 8, 2022 01:41

carlocab commented Aug 8, 2022

View reviewed changes

Formula/python@3.10.rb Outdated Show resolved Hide resolved

carlocab and others added 3 commits August 11, 2022 18:09

python@3.10: use python3 helper method

e8aec98

Co-authored-by: XuehaiPan <XuehaiPan@pku.edu.cn>

python@3.9: update resources

04573e7

python@3.10: update resources

f16e655

carlocab force-pushed the link-all-pythons branch from 1f88fe0 to f16e655 Compare August 11, 2022 10:09

carlocab marked this pull request as ready for review August 11, 2022 10:10

MikeMcQuaid approved these changes Aug 11, 2022

View reviewed changes

carlocab mentioned this pull request Aug 11, 2022

python@3.8: update resources, link into HOMEBREW_PREFIX #107804

Closed

6 tasks

carlocab added CI-long-timeout [DEPRECATED] Use longer GitHub Actions CI timeout. and removed do not merge labels Aug 11, 2022

carlocab added a commit to carlocab/homebrew-core that referenced this pull request Aug 11, 2022

python@3.8: use altinstall target

cc04394

This is a backport of Homebrew#107517 to `python@3.8`.

carlocab mentioned this pull request Aug 11, 2022

python@3.7: update resources, link into HOMEBREW_PREFIX #107816

Closed

6 tasks

cho-m removed the CI-long-timeout [DEPRECATED] Use longer GitHub Actions CI timeout. label Aug 12, 2022

BrewTestBot closed this in b6c57e8 Aug 12, 2022

carlocab deleted the link-all-pythons branch August 12, 2022 04:26

carlocab mentioned this pull request Aug 12, 2022

gitfs: fix test failure #107867

Merged

6 tasks

BrewTestBot pushed a commit that referenced this pull request Aug 12, 2022

python@3.8: use altinstall target

1be735e

This is a backport of #107517 to `python@3.8`.

carlocab added a commit to carlocab/homebrew-core that referenced this pull request Aug 12, 2022

python@3.7: use altinstall target

007a0b3

This is a backport of Homebrew#107517 to `python@3.7`.

carlocab added a commit to carlocab/homebrew-core that referenced this pull request Aug 12, 2022

gitfs: fix python3.9 references.

2e167d5

Fixes a CI failure seen at Homebrew#107517.

BrewTestBot pushed a commit that referenced this pull request Aug 12, 2022

python@3.7: use altinstall target

86ceecf

This is a backport of #107517 to `python@3.7`.

carlocab mentioned this pull request Aug 13, 2022

Fix python3 references in python@3.10-dependent formulae #108008

Closed

jstvz mentioned this pull request Aug 23, 2022

Specify Python minor version in homebrew formula SFDO-Tooling/CumulusCI#3344

Merged

carlocab mentioned this pull request Sep 6, 2022

{Packaging} Update Homebrew template to use python3.10 executable Azure/azure-cli#23788

Merged

3 tasks

carlocab mentioned this pull request Oct 28, 2022

python: make python@3.11 the new default for python3 #114154

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable linking multiple Python formulae #107517

Enable linking multiple Python formulae #107517

carlocab commented Aug 7, 2022 •

edited

Loading

This comment was marked as resolved.

Rylan12 left a comment

carlocab commented Aug 7, 2022 •

edited

Loading

carlocab Aug 8, 2022

carlocab commented Aug 11, 2022 •

edited

Loading

MikeMcQuaid left a comment

MikeMcQuaid Aug 11, 2022

carlocab Aug 11, 2022

MikeMcQuaid Aug 11, 2022

carlocab Aug 11, 2022

MikeMcQuaid Aug 11, 2022

BrewTestBot commented Aug 12, 2022

		If you do not need a specific version of Python, and always want Homebrew's `python3` in your PATH:
		brew install python3

Enable linking multiple Python formulae #107517

Enable linking multiple Python formulae #107517

Conversation

carlocab commented Aug 7, 2022 • edited Loading

This comment was marked as resolved.

Rylan12 left a comment

Choose a reason for hiding this comment

carlocab commented Aug 7, 2022 • edited Loading

carlocab Aug 8, 2022

Choose a reason for hiding this comment

carlocab commented Aug 11, 2022 • edited Loading

MikeMcQuaid left a comment

Choose a reason for hiding this comment

MikeMcQuaid Aug 11, 2022

Choose a reason for hiding this comment

carlocab Aug 11, 2022

Choose a reason for hiding this comment

MikeMcQuaid Aug 11, 2022

Choose a reason for hiding this comment

carlocab Aug 11, 2022

Choose a reason for hiding this comment

MikeMcQuaid Aug 11, 2022

Choose a reason for hiding this comment

BrewTestBot commented Aug 12, 2022

carlocab commented Aug 7, 2022 •

edited

Loading

carlocab commented Aug 7, 2022 •

edited

Loading

carlocab commented Aug 11, 2022 •

edited

Loading