Skip to content

Refine Windows build section, explain config & add how to run #989

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 22, 2022

Conversation

CAM-Gerlach
Copy link
Member

As originally brought up on the Core Dev Discord, problems will result when the build configuration/platform set in the Visual Studio IDE differs from that set on the command line with the PCBuild/build.bat script, and in fact the current defaults are opposite one another (Debug and Win32 for the former, and Release and x64 for the latter). Furthermore, it was noted that the current Windows section in the Setup and Build doc, the listed invocation of the build.bat script uses the Release configuration, when in fact the Debug config is what users developing CPython are likely to want and also doesn't match the default in VS. Finally, it was also noted that the section never specifies how to actually run the built Python, and lacked some other details.

Therefore, this PR:

  • Updates the build.bat invocation to use -c Debug (I considered -d, but went with the former to be more clear and explicit, and consistent with setting other build configs)
  • Adds information about the default build config and how to change them for both the CLI script and VS IDE
  • Adds a note instructing users to rebuild once with the script using a matching build config if it is ever changed
  • Adds a mention in said note to avoid the PGO build options unless specifically needed, per @zooba 's advice

Also, it takes care of some other minor related updates, refinements and fixes in the same section; specifically, it:

  • Updates appropriate mentions of Visual Studio 2017 to refer to Visual Studio more generally, except where stating the minimum version, to avoid confusion that later versions can't be used and ensure link titles are accurate
  • Moves the WSL note to the top of the section rather than the very end, where it is more appropriate (as it refers to the early stages of the process, and is important for the reader to be aware of from the start), and is more easily seen (since WSL is more and more popular these days), and also renders better visually.
  • Makes very minor editorial clarifications/fixes to the prose text (e.g. that the VS IDE can be used is preferred, but not implying that it is needed to "continue development")
  • Adds/uses more specific internal references and external links where appropriate (e.g. pointing specifically to the GfW download page)
  • Uses more descriptive, appropriate and unlikely to collide text for link titles/targets, which are also particularly helpful for a11y
  • Uses appropriate Sphinx roles and other syntax throughout, to format GUI labels, menu selections, verbatim literals, and other things

@ambv ambv merged commit f8b870f into python:main Nov 22, 2022
@ambv
Copy link
Contributor

ambv commented Nov 22, 2022

I like it.

Copy link
Member

@zooba zooba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for missing the merge, I think there's a few more things to fix up.

Comment on lines +248 to +254
.. note:: If you are using the Windows Subsystem for Linux (WSL),
:ref:`clone the repository <checkout>` from a native Windows shell program
like PowerShell or the ``cmd.exe`` command prompt,
and use a build of Git targeted for Windows,
e.g. the `Git for Windows download from the official Git website`_.
Otherwise, Visual Studio will not be able to find all the project's files
and will fail the build.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This note doesn't make a whole lot of sense.

If you're using WSL, you aren't going to be building with Visual Studio. You'll be using autoconf/make/GCC, just like on Linux, because WSL is Linux. So you need to apt/yum/etc. install all your tools and dependencies (and if you're trying to clone to a Windows directory and share the source tree with Windows, you need to modify your EOL settings or it'll clone with CRLF and everything in Linux will barf).

If you're using VS, you are by definition building for Windows, so you will want to install Git for Windows (probably as part of VS), clone on Windows, and run on Windows. No WSL involved.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah; I'm guessing maybe the original intent behind this note was to address users running into problems due to using Git through WSL but wanting to build natively for Windows, but that's not at all clear from the note and I'm not sure how common that is. Should we revise it to just state that using running WSL should build for WSL and/or Windows separately (using the respective compiler/Git)? Or just remove it?

Copy link
Member

@zooba zooba Nov 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably just clarify the note down to (essentially) WSL != Windows and if you're in WSL then <link to Linux section>

Comment on lines +256 to +257
For a concise step by step summary of building Python on Windows,
you can read `Victor Stinner's guide`_.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we have the consice guide in here? Link out to an overly detailed one, but Victor's is exactly what we should have in the devguide.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking something along the same lines; while what's in the devguide is a useful detailed explanation for beginners, I find Victor's guide way more handy as a quick reference (and consult the latter a good deal more often than the former). I didn't want to get this PR even further out of scope, but I was going to propose something like that as a followup. I can either do so as part of the PR addressing your other comments here, or another one after that since its a more substantial and less straightforward change—what do you think is best?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new PR is fine

.. code-block:: dosbatch
.. code-block:: batch

PCbuild\build.bat -c Debug
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PCbuild\build.bat -d is equivalent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah; as mentioned above

I considered [and went back and forth on] -d, but went with the former [-c Debug] to be more clear and explicit, and consistent with setting other build configs [as mentioned in the following paragraph]

but if you feel strongly otherwise, I can go back to -d (its what I would use for the quick step by step reference guide, at least)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There aren't really any other interesting build configs. To do a PGO build, you'd pass --pgo, and you get a release build by omitting -d and --pgo. The fully customisable configuration is very legacy, though occasionally helpful for more intense debugging needs (i.e. not something the devguide is going to cover)

Avoid selecting the ``PGInstrument`` and ``PGUpdate`` configurations,
as these are intended for PGO builds and not for normal development.

You can run the build of Python you've compiled with:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Visual Studio, ensure the python project is set as startup and press F5 (or Ctrl+F5 to run without debugging).

From the command line, a python.bat file is generated in the root of the source tree for the latest build.

Specific builds can be found under their architecture name in the PCbuild folder. For example, a 64-bit Debug build would be at PCbuild\amd64\python_d.exe

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I thought a python.bat would get generated in the root and I tested for it, but due to an unrelated issue with my shell session it wasn't finding it when I tried to invoke it; I can confirm it is indeed there and works as advertised in another shell. I'll revise the section to implement the above points in the followup PR, thanks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trick here is that PowerShell (like Bash) won't search for python.bat in the current directory - you have to type .\python. However, Command Prompt will look in the current directory. So it's easy to get them mixed up.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was actually a lot weirder than that. I was using Git Bash, where it normally does search the CWD for .bat/.cmd scripts, but a borked install of git via Conda in the same shell tab had silently broken that (and a lot of other shell functionality). For some reason, at the time I never through to try the ./python.bat that I'm normally used to running bash, etc. scripts.

@CAM-Gerlach
Copy link
Member Author

Thanks for the expert feedback and detailed review as always, @zooba ; I'm not sure why this PR was merged so quickly before you had a chance to. In any case, I had a couple followup questions on your comments as to how you wanted to handle them, and once I hear from you I can open another PR to implement them, and perhaps a separate followup to add a quick reference and revise/reorganize the rest accordingly to focus just on the specific issues the quick reference doesn't cover (unless you have a better place we can put it right now).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants