Skip to content

Commit

Permalink
Forward port macOS installer updates from 3.7/3.8/3.9 (pythonGH-21132)
Browse files Browse the repository at this point in the history
  • Loading branch information
@ned-deily @arun-mani-j
ned-deily authored and arun-mani-j committed Jul 21, 2020
1 parent 34c9b5d commit 64af912
Show file tree
Hide file tree
Showing 6 changed files with 139 additions and 260 deletions.
242 changes: 55 additions & 187 deletions Mac/BuildScript/README.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,216 +7,84 @@ framework-based Python out-of-tree, installs it in a funny place with
$DESTROOT, massages that installation to remove .pyc files and such, creates
an Installer package from the installation plus other files in ``resources``
and ``scripts`` and placed that on a ``.dmg`` disk image.

For Python 3.4.0, PSF practice is to build two installer variants
for each release.

1. 32-bit-only, i386 and PPC universal, capable on running on all machines
supported by Mac OS X 10.5 through (at least) 10.9::

/path/to/bootstrap/python2.7 build-installer.py \
--sdk-path=/Developer/SDKs/MacOSX10.5.sdk \
--universal-archs=32-bit \
--dep-target=10.5

- builds the following third-party libraries

* NCurses 5.9 (http://bugs.python.org/issue15037)
* SQLite 3.8.11
* XZ 5.0.5

- uses system-supplied versions of third-party libraries

* readline module links with Apple BSD editline (libedit)

- requires ActiveState ``Tcl/Tk 8.4`` (currently 8.4.20) to be installed for building

- recommended build environment:

* Mac OS X 10.5.8 Intel or PPC
* Xcode 3.1.4
* ``MacOSX10.5`` SDK
* ``MACOSX_DEPLOYMENT_TARGET=10.5``
* Apple ``gcc-4.2``
* bootstrap non-framework Python 2.7 for documentation build with
Sphinx (as of 3.4.1)

- alternate build environments:

* Mac OS X 10.6.8 with Xcode 3.2.6
- need to change ``/System/Library/Frameworks/{Tcl,Tk}.framework/Version/Current`` to ``8.4``
* Note Xcode 4.* does not support building for PPC so cannot be used for this build

2. 64-bit / 32-bit, x86_64 and i386 universal, for OS X 10.6 (and later)::
The installer package built on the dmg is a macOS bundle format installer
package. This format is deprecated and is no longer supported by modern
macOS systems; it is usable on macOS 10.6 and earlier systems.
To be usable on newer versions of macOS, the bits in the bundle package
must be assembled in a macOS flat installer package, using current
versions of the pkgbuild and productbuild utilities. To pass macoS
Gatekeeper download quarantine, the final package must be signed
with a valid Apple Developer ID certificate using productsign.
Starting with macOS 10.15 Catalina, Gatekeeper now also requires
that installer packages are submitted to and pass Apple's automated
notarization service using the altool command. To pass notarization,
the binaries included in the package must be built with at least
the macOS 10.9 SDK, mout now be signed with the codesign utility
and executables must opt in to the hardened run time option with
any necessary entitlements. Details of these processes are
available in the on-line Apple Developer Documentation and man pages.

As of 3.8.0 and 3.7.7, PSF practice is to build one installer variants
for each release. Note that as of this writing, no Pythons support
building on a newer version of macOS that will run on older versions
by setting MACOSX_DEPLOYMENT_TARGET. This is because the various
Python C modules do not yet support runtime testing of macOS
feature availability (for example, by using macOS AvailabilityMacros.h
and weak-linking). To build a Python that is to be used on a
range of macOS releases, always build on the oldest release to be
supported; the necessary shared libraries for that release will
normally also be available on later systems, with the occasional
exception such as the removal of 32-bit libraries in macOS 10.15.

build-installer requires Apple Developer tools, either from the
Command Line Tools package or from a full Xcode installation.
You should use the most recent version of either for the operating
system version in use. (One notable exception: on macOS 10.6,
Snow Leopard, use Xcode 3, not Xcode 4 which was released later
in the 10.6 support cycle.)

1. 64-bit, x86_64, for OS X 10.9 (and later)::

/path/to/bootstrap/python2.7 build-installer.py \
--sdk-path=/Developer/SDKs/MacOSX10.6.sdk \
--universal-archs=intel \
--dep-target=10.6
--universal-archs=intel-64 \
--dep-target=10.9

- builds the following third-party libraries

* NCurses 5.9 (http://bugs.python.org/issue15037)
* SQLite 3.8.11
* XZ 5.0.5
* OpenSSL 1.1.1
* Tcl/Tk 8.6
* NCurses
* SQLite
* XZ
* libffi

- uses system-supplied versions of third-party libraries

* readline module links with Apple BSD editline (libedit)

- requires ActiveState Tcl/Tk 8.5.15.1 (or later) to be installed for building

- recommended build environment:

* Mac OS X 10.6.8 (or later)
* Xcode 3.2.6
* ``MacOSX10.6`` SDK
* ``MACOSX_DEPLOYMENT_TARGET=10.6``
* Apple ``gcc-4.2``
* bootstrap non-framework Python 2.7 for documentation build with
Sphinx (as of 3.4.1)

- alternate build environments:

* none. Xcode 4.x currently supplies two C compilers.
``llvm-gcc-4.2.1`` has been found to miscompile Python 3.3.x and
produce a non-functional Python executable. As it appears to be
considered a migration aid by Apple and is not likely to be fixed,
its use should be avoided. The other compiler, ``clang``, has been
undergoing rapid development. While it appears to have become
production-ready in the most recent Xcode 5 releases, the versions
available on the deprecated Xcode 4.x for 10.6 were early releases
and did not receive the level of exposure in production environments
that the Xcode 3 gcc-4.2 compiler has had.


* For Python 2.7.x and 3.2.x, the 32-bit-only installer was configured to
support Mac OS X 10.3.9 through (at least) 10.6. Because it is
believed that there are few systems still running OS X 10.3 or 10.4
and because it has become increasingly difficult to test and
support the differences in these earlier systems, as of Python 3.3.0 the PSF
32-bit installer no longer supports them. For reference in building such
an installer yourself, the details are::

/usr/bin/python build-installer.py \
--sdk-path=/Developer/SDKs/MacOSX10.4u.sdk \
--universal-archs=32-bit \
--dep-target=10.3

- builds the following third-party libraries

* Bzip2
* NCurses
* GNU Readline (GPL)
* SQLite 3
* XZ
* Zlib 1.2.3
* Oracle Sleepycat DB 4.8 (Python 2.x only)

- requires ActiveState ``Tcl/Tk 8.4`` (currently 8.4.20) to be installed for building
* zlib
* bz2

- recommended build environment:

* Mac OS X 10.5.8 PPC or Intel
* Xcode 3.1.4 (or later)
* ``MacOSX10.4u`` SDK (later SDKs do not support PPC G3 processors)
* ``MACOSX_DEPLOYMENT_TARGET=10.3``
* Apple ``gcc-4.0``
* system Python 2.5 for documentation build with Sphinx

- alternate build environments:

* Mac OS X 10.6.8 with Xcode 3.2.6
- need to change ``/System/Library/Frameworks/{Tcl,Tk}.framework/Version/Current`` to ``8.4``

* Mac OS X 10.9.5
* Xcode Command Line Tools 6.2
* ``MacOSX10.9`` SDK
* ``MACOSX_DEPLOYMENT_TARGET=10.9``
* Apple ``clang``


General Prerequisites
---------------------

* No Fink (in ``/sw``) or MacPorts (in ``/opt/local``) or other local
libraries or utilities (in ``/usr/local``) as they could
* No Fink (in ``/sw``) or MacPorts (in ``/opt/local``) or Homebrew or
other local libraries or utilities (in ``/usr/local``) as they could
interfere with the build.

* The documentation for the release is built using Sphinx
because it is included in the installer. For 2.7.x and 3.x.x up to and
including 3.4.0, the ``Doc/Makefile`` uses ``svn`` to download repos of
``Sphinx`` and its dependencies. Beginning with 3.4.1, the ``Doc/Makefile``
assumes there is an externally-provided ``sphinx-build`` and requires at
least Python 2.6 to run. Because of this, it is no longer possible to
build a 3.4.1 or later installer on OS X 10.5 using the Apple-supplied
Python 2.5.

* It is safest to start each variant build with an empty source directory
populated with a fresh copy of the untarred source.
populated with a fresh copy of the untarred source or a source repo.

* It is recommended that you remove any existing installed version of the
Python being built::

sudo rm -rf /Library/Frameworks/Python.framework/Versions/n.n


The Recipe
----------

Here are the steps you need to follow to build a Python installer:

* Run ``build-installer.py``. Optionally you can pass a number of arguments
to specify locations of various files. Please see the top of
``build-installer.py`` for its usage.

Running this script takes some time, it will not only build Python itself
but also some 3th-party libraries that are needed for extensions.

* When done the script will tell you where the DMG image is (by default
somewhere in ``/tmp/_py``).

Building other universal installers
...................................

It is also possible to build a 4-way universal installer that runs on
OS X 10.5 Leopard or later::

/usr/bin/python /build-installer.py \
--dep-target=10.5
--universal-archs=all
--sdk-path=/Developer/SDKs/MacOSX10.5.sdk

This requires that the deployment target is 10.5, and hence
also that you are building on at least OS X 10.5. 4-way includes
``i386``, ``x86_64``, ``ppc``, and ``ppc64`` (G5). ``ppc64`` executable
variants can only be run on G5 machines running 10.5. Note that,
while OS X 10.6 is only supported on Intel-based machines, it is possible
to run ``ppc`` (32-bit) executables unmodified thanks to the Rosetta ppc
emulation in OS X 10.5 and 10.6. The 4-way installer variant must be
built with Xcode 3. It is not regularly built or tested.

Other ``--universal-archs`` options are ``64-bit`` (``x86_64``, ``ppc64``),
and ``3-way`` (``ppc``, ``i386``, ``x86_64``). None of these options
are regularly exercised; use at your own risk.


Testing
-------

Ideally, the resulting binaries should be installed and the test suite run
on all supported OS X releases and architectures. As a practical matter,
that is generally not possible. At a minimum, variant 1 should be run on
a PPC G4 system with OS X 10.5 and at least one Intel system running OS X
10.9, 10.8, 10.7, 10.6, or 10.5. Variant 2 should be run on 10.9, 10.8,
10.7, and 10.6 systems in both 32-bit and 64-bit modes.::

/usr/local/bin/pythonn.n -m test -w -u all,-largefile
/usr/local/bin/pythonn.n-32 -m test -w -u all

Certain tests will be skipped and some cause the interpreter to fail
which will likely generate ``Python quit unexpectedly`` alert messages
to be generated at several points during a test run. These are normal
during testing and can be ignored.

It is also recommend to launch IDLE and verify that it is at least
functional. Double-click on the IDLE app icon in ``/Applications/Python n.n``.
It should also be tested from the command line::

/usr/local/bin/idlen.n

Loading

0 comments on commit 64af912

Please sign in to comment.