python@3.9: use our own tcl-tk

[fxcoudert]

I think we've hit the point where we can't avoid this anymore… Why?

• upstream (cpython) recommends building with recent tcl (8.6 branch): https://bugs.python.org/issue42691
• upstream also explicitly recommends not using Apple's tcl-tk
• macOS ships with an old 8.5 tcl
• that old tcl-tk is actually broken on Big Sur currently (python@3.9: skip test due to Tcl/Tk failure #67242)
• there are other bugs, too

$ tclsh

WARNING: This version of tcl is included in macOS for compatibility with legacy software. 
In future versions of macOS the tcl runtime will not be available by 
default, and may require you to install an additional package.

[carlocab]

$ tclsh

WARNING: This version of tcl is included in macOS for compatibility with legacy software. 
In future versions of macOS the tcl runtime will not be available by 
default, and may require you to install an additional package.

I wonder if it's worth also considering not making tcl-tk keg-only in light of this.

[iMichka]

Just one comment: tcl-tk is quite a big dependency (especially on linux) where it brings in the whole x11 dependency tree. Also, you need python@3.9 for libxcb and xcb-proto, so on linux there is a cyclic dependency. This is an issue I was never able to solve.

This should not influence too much the decision on what is done on Mac, just to let you know that we will disable tcl-tk for Linux.

You'll have also a bunch of people complaining that they wanted to have python@3.9 and are wondering why tcl-tk is being shipped.

[fxcoudert]

Thanks @iMichka for the comments.

You'll have also a bunch of people complaining that they wanted to have python@3.9 and are wondering why tcl-tk is being shipped

Right now we have people complaining that it's broken 😄
At least tcl-tk is not so big (on macOS) compared to python itself

[MikeMcQuaid]

I agree we should either disable Tcl/tk or use our version. I would prefer we not ship this as a dependency but don't mind if I'm in the minority. I also wonder how hard it would be to strip out the Tcl/tk parts into another formula or have a cask for the GUI components?

[iMichka]

Debian ships it as a separate package: https://packages.debian.org/en/sid/python3-tk

[jonchang]

Debian ships it as a separate package: https://packages.debian.org/en/sid/python3-tk

Looking at the file list, it looks like the only thing that is actually compiled is /usr/lib/python3.9/lib-dynload/_tkinter.cpython-39-x86_64-linux-gnu.so. Examining the file on macOS:

% otool -L Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_tkinter.cpython-39-darwin.so
Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_tkinter.cpython-39-darwin.so:
	/System/Library/Frameworks/Tcl.framework/Versions/8.5/Tcl (compatibility version 8.5.0, current version 8.5.9)
	/System/Library/Frameworks/Tk.framework/Versions/8.5/Tk (compatibility version 8.5.0, current version 8.5.9)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.0.0)

suggests that we could use the same approach as libgccjit and create a new python-tk formula that just ships that library and the relevant python files.

[fxcoudert]

I'd rather we keep tkinter in the main python formula, for two reasons:

• it's a standard python module https://docs.python.org/3/library/tkinter.html
• it's present in the system python and all python distributions I know on macOS

I know Debian ships it separately, but they also ship separate packages for libraries, dev tools and headers, built from the same source; it's not how we do things, and I think it would complicate things to do them that way.

As for some data, tcl-tk is basically the same size (55 MB) as all the other dependencies of python. Python itself is 146 MB.

[MikeMcQuaid]

As for some data, tcl-tk is basically the same size (55 MB) as all the other dependencies of python. Python itself is 146 MB.

What size is the full dependency tree it pulls in and how many formulae are there? Given how much depends on Python: this is my concern; we're going to end up force-installing these for everyone who uses anything that uses Python.

[carlocab]

What size is the full dependency tree it pulls in and how many formulae are there?

I believe this is just openssl on macOS, so it shouldn't pull in anything new. On Linux, it seems to pull in a large part of Xorg.

[fxcoudert]

tcl-tk only depends on openssl, which is already required by python.
So we would go from 55 MB deps to 110 MB deps (for a 146 MB python formula);

[gromgit]

On Linux, it seems to pull in a large part of Xorg.

Tk is actually infamous for rolling its own UI stuff. It was only in recent years that Tk enabled Aqua on macOS, but on other Unix platforms, Tk still uses only the base X11 libraries, so on Linux:

$ brew linkage tcl-tk
System libraries:
  /lib/x86_64-linux-gnu/libc.so.6
  /lib/x86_64-linux-gnu/libdl.so.2
  /lib/x86_64-linux-gnu/libm.so.6
  /lib/x86_64-linux-gnu/libpthread.so.0
Homebrew libraries:
  /home/linuxbrew/.linuxbrew/lib/libX11.so.6 (libx11)
  /home/linuxbrew/.linuxbrew/opt/openssl@1.1/lib/libcrypto.so.1.1 (openssl@1.1)
  /home/linuxbrew/.linuxbrew/opt/openssl@1.1/lib/libssl.so.1.1 (openssl@1.1)
  /home/linuxbrew/.linuxbrew/Cellar/tcl-tk/8.6.10_3/lib/libtcl8.6.so (tcl-tk)
  /home/linuxbrew/.linuxbrew/Cellar/tcl-tk/8.6.10_3/lib/libtk8.6.so (tcl-tk)
  /home/linuxbrew/.linuxbrew/lib/libz.so.1 (zlib)
Dependencies with no linkage:
  libxext

The bulk of tcl-tk, 60% of disk usage in fact...is man pages.

[carlocab]

From #67718:

opencascade is using system tcl-tk, causing the same Big Sur bug as with python@3.9:

homebrew-core/Formula/opencascade.rb

Lines 39 to 41 in 93aebb8

	"-D3RDPARTY_TCL_DIR:PATH=#{MacOS.sdk_path_if_needed}/usr",
	"-D3RDPARTY_TCL_INCLUDE_DIR=#{MacOS.sdk_path_if_needed}/usr/include",
	"-D3RDPARTY_TK_INCLUDE_DIR=#{MacOS.sdk_path_if_needed}/usr/include",

[m000]

Next version of Python will be using pkg-config to get the version of Tcl/Tk. The PR is waiting for merge (see
python/cpython#23721), but it can be used in the meantime to simplify the formula. I have implemented this in m000@f4bfef0.

[carlocab]

We've received more reports about the Tcl/Tk bug again recently, so I'd like to see this merged sooner rather than later. Rebasing against master.

[MikeMcQuaid]

tcl-tk only depends on openssl, which is already required by python.
So we would go from 55 MB deps to 110 MB deps (for a 146 MB python formula);

Fine with me given this 👍🏻

[fxcoudert]

10.14 failures:

brew test --retry --verbose hyperkit
brew test --retry --verbose ocrmypdf
brew test --retry --verbose sslyze
brew test --retry --verbose subversion
brew test --retry --verbose swift
brew test --retry --verbose treefrog
brew test --retry --verbose yafc

10.15 adds:

brew test --retry --verbose glyr
brew fetch --retry minidlna
brew fetch --retry mpd
brew test --retry --verbose ocrmypdf
brew test --retry --verbose qxmpp

And 11:

brew test --retry --verbose asuka
brew test --retry --verbose augustus
brew test --retry --verbose aws-google-auth
brew test --retry --verbose creduce
brew test --retry --verbose geant4
brew test --retry --verbose networkit
brew install --only-dependencies oclgrind
brew install oclgrind

[BrewTestBot]

@fxcoudert has triggered a merge.