Skip to content

Refine Windows build section, explain config & add how to run #989

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 22, 2022
Merged

Refine Windows build section, explain config & add how to run #989

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4884ed5
Add -c Debug & note on changing build config/platform for Windows build
CAM-Gerlach Nov 22, 2022
57f9598
Use Sphinx syntax and links w/descriptive titles in section
CAM-Gerlach Nov 22, 2022
c45525f
Move WSL note to the top of the section where it is more easily noticed
CAM-Gerlach Nov 22, 2022
82ea4e9
Add a line explaining how to run the built Python on Windows
CAM-Gerlach Nov 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 63 additions & 30 deletions getting-started/setup-building.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,47 +245,80 @@ still build properly).
Windows
-------

For a quick guide to building you can read `this documentation`_ from Victor
Stinner.

All current versions of Python can be built using Microsoft Visual Studio 2017
or later. You can download
and use any of the free or paid versions of `Visual Studio 2017`_.

When installing Visual Studio 2017, select the **Python development** workload
and the optional **Python native development tools** component to obtain all of
the necessary build tools. If you do not already have git installed, you can
find git for Windows on the **Individual components** tab of the installer.
.. note:: If you are using the Windows Subsystem for Linux (WSL),
:ref:`clone the repository <checkout>` from a native Windows shell program
like PowerShell or the ``cmd.exe`` command prompt,
and use a build of Git targeted for Windows,
e.g. the `Git for Windows download from the official Git website`_.
Otherwise, Visual Studio will not be able to find all the project's files
and will fail the build.
Comment on lines +248 to +254
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This note doesn't make a whole lot of sense.

If you're using WSL, you aren't going to be building with Visual Studio. You'll be using autoconf/make/GCC, just like on Linux, because WSL is Linux. So you need to apt/yum/etc. install all your tools and dependencies (and if you're trying to clone to a Windows directory and share the source tree with Windows, you need to modify your EOL settings or it'll clone with CRLF and everything in Linux will barf).

If you're using VS, you are by definition building for Windows, so you will want to install Git for Windows (probably as part of VS), clone on Windows, and run on Windows. No WSL involved.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah; I'm guessing maybe the original intent behind this note was to address users running into problems due to using Git through WSL but wanting to build natively for Windows, but that's not at all clear from the note and I'm not sure how common that is. Should we revise it to just state that using running WSL should build for WSL and/or Windows separately (using the respective compiler/Git)? Or just remove it?

Copy link
Member

@zooba zooba Nov 23, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably just clarify the note down to (essentially) WSL != Windows and if you're in WSL then <link to Linux section>


For a concise step by step summary of building Python on Windows,
you can read `Victor Stinner's guide`_.
Comment on lines +256 to +257
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we have the consice guide in here? Link out to an overly detailed one, but Victor's is exactly what we should have in the devguide.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking something along the same lines; while what's in the devguide is a useful detailed explanation for beginners, I find Victor's guide way more handy as a quick reference (and consult the latter a good deal more often than the former). I didn't want to get this PR even further out of scope, but I was going to propose something like that as a followup. I can either do so as part of the PR addressing your other comments here, or another one after that since its a more substantial and less straightforward change—what do you think is best?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new PR is fine


All supported versions of Python can be built
using Microsoft Visual Studio 2017 or later.
You can download and use any of the free or paid versions of `Visual Studio`_.

When installing it, select the :guilabel:`Python development` workload
and the optional :guilabel:`Python native development tools` component
to obtain all of the necessary build tools.
You can find Git for Windows on the :guilabel:`Individual components` tab
if you don't already have it installed.

.. note:: If you want to build MSI installers, be aware that the build toolchain
for them has a dependency on the Microsoft .NET Framework Version 3.5 (which
may not be configured on recent versions of Windows, such as Windows 10). If
you are building on a recent Windows version, use the Control Panel (Programs
| Programs and Features | Turn Windows Features on or off) and ensure that the
entry ".NET Framework 3.5 (includes .NET 2.0 and 3.0)" is enabled.
for them has a dependency on the Microsoft .NET Framework Version 3.5
(which may not be included on recent versions of Windows, such as Windows 10).
If you are building on a recent Windows version, use the Control Panel
(:menuselection:`Programs --> Programs and Features --> Turn Windows Features on or off`)
and ensure that the entry
:guilabel:`.NET Framework 3.5 (includes .NET 2.0 and 3.0)` is enabled.

Your first build should use the command line to ensure any external dependencies
are downloaded:

.. code-block:: dosbatch
.. code-block:: batch

PCbuild\build.bat -c Debug
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PCbuild\build.bat -d is equivalent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah; as mentioned above

I considered [and went back and forth on] -d, but went with the former [-c Debug] to be more clear and explicit, and consistent with setting other build configs [as mentioned in the following paragraph]

but if you feel strongly otherwise, I can go back to -d (its what I would use for the quick step by step reference guide, at least)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There aren't really any other interesting build configs. To do a PGO build, you'd pass --pgo, and you get a release build by omitting -d and --pgo. The fully customisable configuration is very legacy, though occasionally helpful for more intense debugging needs (i.e. not something the devguide is going to cover)


The above command line build uses the ``-c Debug`` argument
to build in the ``Debug`` configuration,
which enables checks and assertions helpful for developing Python.
By default, it builds in the ``Release`` configuration
and for the 64-bit ``x64`` platform rather than 32-bit ``Win32``;
use ``-c`` and ``-p`` to control build config and platform, respectively.

After this build succeeds, you can open the ``PCbuild\pcbuild.sln`` solution
in the Visual Studio IDE to continue development, if you prefer.
When building in Visual Studio,
make sure to select build settings that match what you used with the script
(the :guilabel:`Debug` configuration and the :guilabel:`x64` platform)
from the dropdown menus in the toolbar.

.. note::

If you need to change the build configuration or platform,
build once with the ``build.bat`` script set to those options first
before building with them in VS to ensure all files are rebuilt properly,
or you may encouter errors when loading modules that were not rebuilt.

Avoid selecting the ``PGInstrument`` and ``PGUpdate`` configurations,
as these are intended for PGO builds and not for normal development.

You can run the build of Python you've compiled with:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Visual Studio, ensure the python project is set as startup and press F5 (or Ctrl+F5 to run without debugging).

From the command line, a python.bat file is generated in the root of the source tree for the latest build.

Specific builds can be found under their architecture name in the PCbuild folder. For example, a 64-bit Debug build would be at PCbuild\amd64\python_d.exe

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I thought a python.bat would get generated in the root and I tested for it, but due to an unrelated issue with my shell session it wasn't finding it when I tried to invoke it; I can confirm it is indeed there and works as advertised in another shell. I'll revise the section to implement the above points in the followup PR, thanks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trick here is that PowerShell (like Bash) won't search for python.bat in the current directory - you have to type .\python. However, Command Prompt will look in the current directory. So it's easy to get them mixed up.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was actually a lot weirder than that. I was using Git Bash, where it normally does search the CWD for .bat/.cmd scripts, but a borked install of git via Conda in the same shell tab had silently broken that (and a lot of other shell functionality). For some reason, at the time I never through to try the ./python.bat that I'm normally used to running bash, etc. scripts.


PCbuild\build.bat
.. code-block:: batch

After this build succeeds, you can open the ``PCbuild\pcbuild.sln`` solution in
Visual Studio to continue development.
PCbuild\amd64\python_d.exe

See the `readme`_ for more details on what other software is necessary and how
to build.
See the `PCBuild readme`_ for more details on what other software is necessary
and how to build.

.. note:: If you are using the Windows Subsystem for Linux (WSL), clone the
repository from a native Windows terminal program like cmd.exe command prompt
or PowerShell as well as use a build of git targeted for Windows, e.g., the
official one from `<https://git-scm.com>`_. Otherwise, Visual Studio will
not be able to find all the project's files and will fail the build.
.. _Victor Stinner's guide: https://cpython-core-tutorial.readthedocs.io/en/latest/build_cpython_windows.html
.. _Visual Studio: https://visualstudio.microsoft.com/
.. _PCBuild readme: https://github.com/python/cpython/blob/main/PCbuild/readme.txt
.. _Git for Windows download from the official Git website: https://git-scm.com/download/win

.. _this documentation: https://cpython-core-tutorial.readthedocs.io/en/latest/build_cpython_windows.html
.. _Visual Studio 2017: https://visualstudio.microsoft.com/
.. _readme: https://github.com/python/cpython/blob/main/PCbuild/readme.txt

.. _build-dependencies:

Expand Down