Python build scripts: 2.x or 3.x?

[sanmai-NL]

I am about to contribute to src/bootstrap/bootstrap.py, so I checked whether it is supposed to run under Python 2.x or 3.x. I concluded that it is Python 3.x, since there is Python 3 code in there. ☂ Yet README.md says the dependency is on Python 2.x.

1. I suppose README.md should be corrected?
2. Additionally, in order to prevent confusion and compatibility problems, the shebang in all Python script files (i.e., a subset of all Python files) should explicitly refer to the required major Python version like so:
   .

#!/usr/bin/env python3

.
3. Finally, if no final decision has been made as to what version of Python and/or what version of the CPython implementation is required, I have the following order of preference:
CPython >=3.3 > CPython >=2.7 > Combined code base of CPython >=3.3 & >=2.7

Consider this official note that basically explains why Python 3 is preferred. I have found that src/bootstrap/bootstrap.py can be refactored to simpler and shorter code, especially when the CPython >=3.3 standard library is available. (Not that I am happy about the CPython dependency for Rust building in general.)

This official release engineering page states that CPython 3.2 and below are EOL.

[hanna-kruppe]

We don't have a real policy of which Python version to use. LLVM requires Python 2.7, which basically settles the question of minimum supported version. The only variable is whether we make an effort to be 3.x-compatible so that we can upgrade easily once that's possible. Again there's no official policy, some scripts try and some don't, probably depending on the mood of the authors at the time of writing.

[hanna-kruppe]

Aside: I have a nagging feeling that, since we never run LLVM tests, we may get away with simply not passing any Python to LLVM. I don't care enough to wrestle the build system for that, but if you want to see Rust built with 3.x that's probably the best way forward.

[sanmai-NL]

According to #32992, Python 3.5 would be required by LLDB though. Most importantly, the Rust build script I was reporting about seems more important than LLVM indeed, and requires Python 3.x at present.

Wouldn't it be easier in the end to distribute signed native executables to build Rust instead of these scripts? Dropping Python dependencies as soon as possible is optimal.

[hanna-kruppe]

I can't comment on LLDB, but how does bootstrap.py require 3.x? The line you linked to, print("downloading " + url), is 3.x-compatible but also works on 2.7 (in contrast with print("downloading", url) which would require 3.x or from __future__ import print_function). I also seem to recall using rustbuild with Python 2.7.

[sanmai-NL]

Yes, I claimed that incorrectly. I will proceed patching some smaller things in bootstrap.py then in a Python 2.7.11 compatible way.

Your comment spurred a possibly interesting small investigation, because it surprised me.

print("downloading " + url)
print("downloading " + url,)
print["downloading " + url]

all work in both Python 2 and 3, but here everything after print is confusingly lexed as an expression. This document hints at the relevant ‘exception’ in the lexical grammar. E.g.

print"downloading " + url

does not work.

Migration to Python 3?

Using the official CPython 2 to 3 conversion tool

2to3 --print-function rust/

on the whole Rust code base results in only minimal proposed patches. It may be worthwhile to support both CPython 2 and 3, and after a while migrate fully to CPython 3, by correcting the shebang across all Python scripts to begin with.

[caipre]

I believe we require python 2.7 explicitly: https://github.com/rust-lang/rust/blob/master/configure#L723-L730

[alexcrichton]

Thanks for the report! Currently the situation is that we basically require whatever version of Python that LLVM requires, which as mentioned before we believe is 2.7 at this time. We strive to minimize the amount of python that we ourselves actively run in-tree wherever possible, for example many scripts in src/etc are very old or only run occasionally and not part of a normal build.

If we're writing our own script then we have no reason to not support 2 and 3 wherever possible, but in theory the scripts are so small and the tasks are so simple that it should be easy to convert one way or another.

So to answer the question of this issue, it's basically both! I'm gonna close this for now (as I believe we're compatible with the 2.7 requirement that we have for now), but feel free to ask any follow-up questions!