Added windows support to hermetic toolchains #628

UebelAndre · 2022-02-25T21:35:32Z

PR Checklist

Please check if your PR fulfills the following requirements:

Tests for the changes have been added (for bug fixes / features)
Docs have been added / updated (for bug fixes / features)

PR Type

What kind of change does this PR introduce?

Bugfix
Feature (please, look at the "Scope of the project" section in the README.md file)
Code style update (formatting, local variables)
Refactoring (no functional changes, no api changes)
Build related changes
CI related changes
Documentation content changes
Other... Please describe:

What is the current behavior?

Issue Number: N/A

What is the new behavior?

Does this PR introduce a breaking change?

Yes
No

Other information

thundergolfer · 2022-03-01T08:33:44Z

👋 Thanks for getting on this.

I've wrangled my Macbook's Windows 10 VM, and installed this PR using local_repository.

(Sidebar: if anyone has a good guide on how to make Windows VMs on OSX not an awful experience, let me know. I'm using Virtualbox and throwing RAM (32) and CPU (8) at this thing and it's still super sluggish. Graphics issues?)

I'm hitting the following minor problem when I run the PR:

INFO: Repository python310_x86_64-pc-windows-msvc instantiated at:
  C:/users/user/source/repos/rulespythondev/WORKSPACE.bazel:14:27: in <toplevel>
  C:/users/user/_bazel_user/3bdyrqkj/external/rules_python/python/repositories.bzl:178:26: in python_register_toolchains
Repository rule python_repository defined at:
  C:/users/user/_bazel_user/3bdyrqkj/external/rules_python/python/repositories.bzl:134:36: in <toplevel>
WARNING: Download from https://github.com/indygreg/python-build-standalone/releases/download/20220227/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Checksum was a293c5838dd9c8438a84372fb95dda9752df63928a8a2ae516438f187f89567d but wanted 37e647b67a052ba576e8455d82e472ee37e8095b35387d49efe9f63a38d3f6d0
ERROR: An error occurred during the fetch of repository 'python310_x86_64-pc-windows-msvc':
   Traceback (most recent call last):
        File "C:/users/user/_bazel_user/3bdyrqkj/external/rules_python/python/repositories.bzl", line 47, column 48, in _python_repository_impl
                download_result = rctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/indygreg/python-build-standalone/releases/download/20220227/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz] to C:/users/user/_bazel_user/3bdyrqkj/external/python310_x86_64-pc-windows-msvc/temp15347681220738251273/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz: Checksum was a293c5838dd9c8438a84372fb95dda9752df63928a8a2ae516438f187f89567d but wanted 37e647b67a052ba576e8455d82e472ee37e8095b35387d49efe9f63a38d3f6d0
ERROR: C:/users/user/source/repos/rulespythondev/WORKSPACE.bazel:14:27: fetching python_repository rule //external:python310_x86_64-pc-windows-msvc: Traceback (most recent  call last):
        File "C:/users/user/_bazel_user/3bdyrqkj/external/rules_python/python/repositories.bzl", line 47, column 48, in _python_repository_impl
                download_result = rctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/indygreg/python-build-standalone/releases/download/20220227/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz] to C:/users/user/_bazel_user/3bdyrqkj/external/python310_x86_64-pc-windows-msvc/temp15347681220738251273/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz: Checksum was a293c5838dd9c8438a84372fb95dda9752df63928a8a2ae516438f187f89567d but wanted 37e647b67a052ba576e8455d82e472ee37e8095b35387d49efe9f63a38d3f6d0
ERROR: C:/users/user/source/repos/rulespythondev/BUILD.bazel:1:10: //:foo depends on @python310_x86_64-pc-windows-msvc//:python_runtimes in repository @python310_x86_64-pc-windows-msvc which failed to fetch. no such package '@python310_x86_64-pc-windows-msvc//': java.io.IOException: Error downloading [https://github.com/indygreg/python-build-standalone/releases/download/20220227/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz] to C:/users/user/_bazel_user/3bdyrqkj/external/python310_x86_64-pc-windows-msvc/temp15347681220738251273/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz: Checksum was a293c5838dd9c8438a84372fb95dda9752df63928a8a2ae516438f187f89567d but wanted 37e647b67a052ba576e8455d82e472ee37e8095b35387d49efe9f63a38d3f6d0
ERROR: Analysis of target '//:foo' failed; build aborted: Analysis failed
INFO: Elapsed time: 12.891s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (34 packages loaded, 121 targets configured)
FAILED: Build did NOT complete successfully (34 packages loaded, 121 targets configured)

Looks like a straightforward hash mismatch.

thundergolfer · 2022-03-01T08:49:31Z

python/private/versions.bzl

 # buildifier: disable=unsorted-dict-items
 TOOL_VERSIONS = {
    "3.8.12": {
-        "x86_64-apple-darwin": "sha384-es0kCVBb4q5xSC09lOw83TKXtR6qdt0NeU56JtK7Y5M5V784k9MM2q8leE3QWGH6",
-        "x86_64-unknown-linux-gnu": "sha384-61BjosOUm+Uo2yDtkLcRqbSIAXIPiHKmvySGO2J/GSOHXkj5QBf4cwzrDuOWR0H2",
+        "x86_64-apple-darwin": "sha256-wmUK2QuFpdJRh5yvGHG9zgKtLGC0e3rBt2aRJ4tOto8=",


Can we provide these integrity strings as 'standard' hex sha256 strings? I see that the docs say use "Subresource Integrity format" but it seems like a small usability issue that the source code integrity strings are in a different format to what Bazel spits out in logs.

Update: Just tried it, and you unfortunately cannot. Disregard my comment.

I see no reason why these couldn't become sha256 values. It would just require updating the attribute in repositories.bzl I leave this decision up to @f0rmiga #618 (comment)

python/private/versions.bzl

thundergolfer · 2022-03-01T09:04:37Z

Fixing the checksum locally has the basics working on my Windows VM 🎉

$ bazel run //:foo
Loading:
Loading: 0 packages loaded
Analyzing: target //:foo (0 packages loaded, 0 targets configured)
INFO: Analyzed target //:foo (23 packages loaded, 84 targets configured).
INFO: Found 1 target...
[1 / 6] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[5 / 7] PythonZipper foo.zip; 1s local
Target //:foo up-to-date:
  bazel-bin/foo.exe
  bazel-bin/foo.zip
INFO: Elapsed time: 3.301s, Critical Path: 2.74s
INFO: 2 processes: 1 internal, 1 local.
INFO: Build completed successfully, 2 total actions
INFO: Running command line: bazel-bin/foo.exe
INFO: Build completed successfully, 2 total actions
sys.version='3.10.2 (main, Feb 27 2022, 18:43:34) [MSC v.1929 64 bit (AMD64)]'
sys.executable='C:\\Users\\User\\AppData\\Local\\Temp\\Bazel.runfiles_pj0rlg5e\\runfiles\\python310_x86_64-pc-windows-msvc\\python.exe'
hello world

UebelAndre · 2022-03-01T14:54:30Z

@thundergolfer in case it's not obvious, this PR is targeting #618

UebelAndre · 2022-03-01T14:55:19Z

I'm hitting the following minor problem when I run the PR:

INFO: Repository python310_x86_64-pc-windows-msvc instantiated at:
  C:/users/user/source/repos/rulespythondev/WORKSPACE.bazel:14:27: in <toplevel>
  C:/users/user/_bazel_user/3bdyrqkj/external/rules_python/python/repositories.bzl:178:26: in python_register_toolchains
Repository rule python_repository defined at:
  C:/users/user/_bazel_user/3bdyrqkj/external/rules_python/python/repositories.bzl:134:36: in <toplevel>
WARNING: Download from https://github.com/indygreg/python-build-standalone/releases/download/20220227/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Checksum was a293c5838dd9c8438a84372fb95dda9752df63928a8a2ae516438f187f89567d but wanted 37e647b67a052ba576e8455d82e472ee37e8095b35387d49efe9f63a38d3f6d0
ERROR: An error occurred during the fetch of repository 'python310_x86_64-pc-windows-msvc':
   Traceback (most recent call last):
        File "C:/users/user/_bazel_user/3bdyrqkj/external/rules_python/python/repositories.bzl", line 47, column 48, in _python_repository_impl
                download_result = rctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/indygreg/python-build-standalone/releases/download/20220227/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz] to C:/users/user/_bazel_user/3bdyrqkj/external/python310_x86_64-pc-windows-msvc/temp15347681220738251273/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz: Checksum was a293c5838dd9c8438a84372fb95dda9752df63928a8a2ae516438f187f89567d but wanted 37e647b67a052ba576e8455d82e472ee37e8095b35387d49efe9f63a38d3f6d0
ERROR: C:/users/user/source/repos/rulespythondev/WORKSPACE.bazel:14:27: fetching python_repository rule //external:python310_x86_64-pc-windows-msvc: Traceback (most recent  call last):
        File "C:/users/user/_bazel_user/3bdyrqkj/external/rules_python/python/repositories.bzl", line 47, column 48, in _python_repository_impl
                download_result = rctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/indygreg/python-build-standalone/releases/download/20220227/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz] to C:/users/user/_bazel_user/3bdyrqkj/external/python310_x86_64-pc-windows-msvc/temp15347681220738251273/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz: Checksum was a293c5838dd9c8438a84372fb95dda9752df63928a8a2ae516438f187f89567d but wanted 37e647b67a052ba576e8455d82e472ee37e8095b35387d49efe9f63a38d3f6d0
ERROR: C:/users/user/source/repos/rulespythondev/BUILD.bazel:1:10: //:foo depends on @python310_x86_64-pc-windows-msvc//:python_runtimes in repository @python310_x86_64-pc-windows-msvc which failed to fetch. no such package '@python310_x86_64-pc-windows-msvc//': java.io.IOException: Error downloading [https://github.com/indygreg/python-build-standalone/releases/download/20220227/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz] to C:/users/user/_bazel_user/3bdyrqkj/external/python310_x86_64-pc-windows-msvc/temp15347681220738251273/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-static-install_only.tar.gz: Checksum was a293c5838dd9c8438a84372fb95dda9752df63928a8a2ae516438f187f89567d but wanted 37e647b67a052ba576e8455d82e472ee37e8095b35387d49efe9f63a38d3f6d0
ERROR: Analysis of target '//:foo' failed; build aborted: Analysis failed
INFO: Elapsed time: 12.891s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (34 packages loaded, 121 targets configured)
FAILED: Build did NOT complete successfully (34 packages loaded, 121 targets configured)

Looks like a straightforward hash mismatch.

@thundergolfer perhaps you're running into indygreg/python-build-standalone#116 (comment)

UebelAndre · 2022-03-02T15:48:16Z

Looks like a straightforward hash mismatch.

Updated the checksums

f0rmiga

Awesome! Thanks for working on the Windows portion.

python/private/versions.bzl

f0rmiga · 2022-03-02T19:37:26Z

python/repositories.bzl

+    is_host_windows = "win" in rctx.os.name
+    is_target_windows = "windows" in rctx.attr.platform
+    python_version = rctx.attr.python_version
+    build = "static-install_only" if is_target_windows else "install_only"


I'm not a Windows user. Would a static binary for Windows have the same limitation as a fully static binary for Linux that a shared object (i.e. a native Python extension) cannot be loaded at runtime (not able to do dlopen)?

I'm not sure myself, but this was the only artifact that worked. My thought was to hopefully see something merged that works in some cases and fix other issues as they're reported.

Well, this could possibly be a blocker for merging my PR into main. I'll take this for now, but will have to investigate before releasing this feature.

f0rmiga · 2022-03-02T19:41:26Z

python/repositories.bzl

+        include = [
+            "bin/**",
+            "DLLs/**",
+            "extensions/**",
+            "include/**",
+            "lib/**",
+            "libs/**",
+            "Scripts/**",
+            "share/**",
+        ],


A select with the platforms would be nicer here.

python/repositories.bzl

Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>

* Added windows support to hermetic toolchains * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>

Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: compile zstd if missing Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: buildifier Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: make python_repositories reproducible Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> rename: python_repositories -> python_repository Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: linter Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: make interpreter files publicly visible Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: add files to py_runtime Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Account for some platforms not having all versions Added windows support to hermetic toolchains (bazelbuild#628) * Added windows support to hermetic toolchains * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> refactor: simplify logic for release urls Also, added a helper target to print the release hashes. Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: Provide a host platform alias (bazelbuild#635) * feat: Provide a host platform alias This lets users and repository rules access the interpreter for whatever host the repository is running on. * Apply suggestions from code review Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: files excludes Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: macOS dislikes --recursive Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: buildifier issues Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Allow previous indygreg releases (bazelbuild#636) This gives us more python patch versions fix: put back zstd support for older releases Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: hash calculator Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: use hermetic interpreter with pip_parse and pip_install Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: add missing attrs back for zstd Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: expose zstd attributes Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: normalize OS names Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: linting issues Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: support windows in the aliases Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: linting issues Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: windows python.exe instead of python3.exe Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: use consts for OS names Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: always use latest toolchain for test Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: expose versions.bzl Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> refactor: move toolchain tests out of private Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: acceptance tests for the toolchains Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: rewrite test in py to work on windows Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: README example Co-authored-by: UebelAndre <github@uebelandre.com> fix: use toolchain to run acceptance tests Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: use matrix for acceptance tests Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: support acceptance_tests on windows Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: alias for pip Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix?: include call to windows cmd Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Fix windows acceptance tests (bazelbuild#641) * Fix windows acceptance tests * test * todo: remove Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> refactor: polishing Windows testing support Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: unset py2_runtime Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> rename: host -> resolved_interpreter Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> doc: add reference to quirks in python-build-standalone Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: allow a distutils.cfg to be passed Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: buildifier (again) Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Minor code review suggestions

Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: compile zstd if missing Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: buildifier Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: make python_repositories reproducible Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> rename: python_repositories -> python_repository Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: linter Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: make interpreter files publicly visible Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: add files to py_runtime Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Account for some platforms not having all versions Added windows support to hermetic toolchains (#628) * Added windows support to hermetic toolchains * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> refactor: simplify logic for release urls Also, added a helper target to print the release hashes. Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: Provide a host platform alias (#635) * feat: Provide a host platform alias This lets users and repository rules access the interpreter for whatever host the repository is running on. * Apply suggestions from code review Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: files excludes Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: macOS dislikes --recursive Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: buildifier issues Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Allow previous indygreg releases (#636) This gives us more python patch versions fix: put back zstd support for older releases Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: hash calculator Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: use hermetic interpreter with pip_parse and pip_install Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: add missing attrs back for zstd Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: expose zstd attributes Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: normalize OS names Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: linting issues Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: support windows in the aliases Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: linting issues Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: windows python.exe instead of python3.exe Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: use consts for OS names Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: always use latest toolchain for test Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: expose versions.bzl Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> refactor: move toolchain tests out of private Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: acceptance tests for the toolchains Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: rewrite test in py to work on windows Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: README example Co-authored-by: UebelAndre <github@uebelandre.com> fix: use toolchain to run acceptance tests Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: use matrix for acceptance tests Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: support acceptance_tests on windows Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: alias for pip Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix?: include call to windows cmd Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Fix windows acceptance tests (#641) * Fix windows acceptance tests * test * todo: remove Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> refactor: polishing Windows testing support Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: unset py2_runtime Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> rename: host -> resolved_interpreter Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> doc: add reference to quirks in python-build-standalone Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> feat: allow a distutils.cfg to be passed Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> fix: buildifier (again) Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Minor code review suggestions

* feat: cpython toolchains for linux and macos Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: compile zstd if missing Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: buildifier Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: make python_repositories reproducible Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * rename: python_repositories -> python_repository Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: linter Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: make interpreter files publicly visible Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: add files to py_runtime Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Account for some platforms not having all versions * Added windows support to hermetic toolchains (#628) * Added windows support to hermetic toolchains * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Update python/repositories.bzl Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: simplify logic for release urls Also, added a helper target to print the release hashes. Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: Provide a host platform alias (#635) * feat: Provide a host platform alias This lets users and repository rules access the interpreter for whatever host the repository is running on. * Apply suggestions from code review Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: files excludes Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: macOS dislikes --recursive Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: buildifier issues Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Allow previous indygreg releases (#636) This gives us more python patch versions * fix: put back zstd support for older releases Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: hash calculator Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: use hermetic interpreter with pip_parse and pip_install Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: add missing attrs back for zstd Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: expose zstd attributes Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: normalize OS names Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: linting issues Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: support windows in the aliases Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: linting issues Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: windows python.exe instead of python3.exe Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: use consts for OS names Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: always use latest toolchain for test Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: expose versions.bzl Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: move toolchain tests out of private Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: acceptance tests for the toolchains Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: rewrite test in py to work on windows Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: README example Co-authored-by: UebelAndre <github@uebelandre.com> * fix: use toolchain to run acceptance tests Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: use matrix for acceptance tests Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: support acceptance_tests on windows Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: alias for pip Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix?: include call to windows cmd Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Fix windows acceptance tests (#641) * Fix windows acceptance tests * test * todo: remove Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: polishing Windows testing support Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: unset py2_runtime Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * rename: host -> resolved_interpreter Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * doc: add reference to quirks in python-build-standalone Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: allow a distutils.cfg to be passed Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: buildifier (again) Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * Minor code review suggestions (#642) * Minor code review suggestions * Apply suggestions from code review Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: depset concat Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Co-authored-by: Alex Eagle <alex@aspect.dev> Co-authored-by: UebelAndre <github@uebelandre.com>

UebelAndre force-pushed the cython branch 5 times, most recently from 92bf7dd to bb6eee6 Compare February 27, 2022 17:40

UebelAndre marked this pull request as ready for review February 27, 2022 17:47

UebelAndre requested review from brandjon and lberki as code owners February 27, 2022 17:47

thundergolfer requested review from thundergolfer and removed request for brandjon and lberki March 1, 2022 07:40

thundergolfer reviewed Mar 1, 2022

View reviewed changes

python/private/versions.bzl Show resolved Hide resolved

Added windows support to hermetic toolchains

227aa3d

UebelAndre force-pushed the cython branch from bb6eee6 to 227aa3d Compare March 2, 2022 15:47

f0rmiga requested changes Mar 2, 2022

View reviewed changes

UebelAndre and others added 4 commits March 2, 2022 12:07

Update python/repositories.bzl

09c57bd

Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>

Update python/repositories.bzl

8a53ef4

Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>

Update python/repositories.bzl

3ce5c91

Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>

Update python/repositories.bzl

e063c20

Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>

f0rmiga approved these changes Mar 2, 2022

View reviewed changes

f0rmiga merged commit 0beb5fb into bazelbuild:f0rmiga/cpython-toolchain Mar 2, 2022

UebelAndre deleted the cython branch March 2, 2022 21:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added windows support to hermetic toolchains #628

Added windows support to hermetic toolchains #628

UebelAndre commented Feb 25, 2022

thundergolfer commented Mar 1, 2022 •

edited

Loading

thundergolfer Mar 1, 2022 •

edited

Loading

UebelAndre Mar 1, 2022

thundergolfer commented Mar 1, 2022 •

edited

Loading

UebelAndre commented Mar 1, 2022

UebelAndre commented Mar 1, 2022

UebelAndre commented Mar 2, 2022

f0rmiga left a comment

f0rmiga Mar 2, 2022

UebelAndre Mar 2, 2022

f0rmiga Mar 2, 2022

f0rmiga Mar 2, 2022

Added windows support to hermetic toolchains #628

Added windows support to hermetic toolchains #628

Conversation

UebelAndre commented Feb 25, 2022

PR Checklist

PR Type

What is the current behavior?

What is the new behavior?

Does this PR introduce a breaking change?

Other information

thundergolfer commented Mar 1, 2022 • edited Loading

thundergolfer Mar 1, 2022 • edited Loading

Choose a reason for hiding this comment

UebelAndre Mar 1, 2022

Choose a reason for hiding this comment

thundergolfer commented Mar 1, 2022 • edited Loading

UebelAndre commented Mar 1, 2022

UebelAndre commented Mar 1, 2022

UebelAndre commented Mar 2, 2022

f0rmiga left a comment

Choose a reason for hiding this comment

f0rmiga Mar 2, 2022

Choose a reason for hiding this comment

UebelAndre Mar 2, 2022

Choose a reason for hiding this comment

f0rmiga Mar 2, 2022

Choose a reason for hiding this comment

f0rmiga Mar 2, 2022

Choose a reason for hiding this comment

thundergolfer commented Mar 1, 2022 •

edited

Loading

thundergolfer Mar 1, 2022 •

edited

Loading

thundergolfer commented Mar 1, 2022 •

edited

Loading