Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test native Windows on CI #1745

Merged
merged 29 commits into from
Nov 29, 2023
Merged
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
2fd79f4
Add native Windows test jobs to CI matrix
EliahKagan Oct 15, 2023
6e477e3
Add xfail marks for IndexFile.from_tree failures
EliahKagan Nov 14, 2023
cd9d7a9
Mark test_clone_command_injection xfail on Windows
EliahKagan Nov 15, 2023
f72e282
Mark test_diff_submodule xfail on Windows
EliahKagan Nov 15, 2023
42a3d74
Mark TestSubmodule.test_rename xfail on Windows
EliahKagan Nov 15, 2023
4abab92
Mark test_conditional_includes_from_git_dir xfail on Windows
EliahKagan Nov 15, 2023
799c853
Improve ordering/grouping of a few imports
EliahKagan Nov 16, 2023
b284ad7
Mark test_create_remote_unsafe_url_allowed xfail on Windows
EliahKagan Nov 16, 2023
61d1fba
Mark unsafe-options "allowed" tests xfail on Windows
EliahKagan Nov 16, 2023
ad07ecb
Show PATH on CI
EliahKagan Nov 22, 2023
2784e40
Show bash and other WSL-relevant info but not PATH
EliahKagan Nov 22, 2023
9717b8d
Install WSL system on CI for hook tests
EliahKagan Nov 17, 2023
5d11394
Fix and expand bash.exe xfail marks on hook tests
EliahKagan Nov 23, 2023
b215357
Simplify/clarify bash.exe check for hook tests; do it only once
EliahKagan Nov 24, 2023
cabb572
Temporarily don't install WSL system to test xfail
EliahKagan Nov 24, 2023
2875ffa
Put back WSL on Windows CI; pare down debug info
EliahKagan Nov 24, 2023
0f8cd4c
Treat XPASS status as a test failure
EliahKagan Nov 24, 2023
82c361e
Correct TestSubmodule.test_rename xfail condition
EliahKagan Nov 24, 2023
0ae5dd1
Revert "Treat XPASS status as a test failure"
EliahKagan Nov 24, 2023
0b7ee17
Refine TestSubmodule.test_rename xfail condition
EliahKagan Nov 25, 2023
8621e89
Reword comment in _WinBashStatus.check for clarity
EliahKagan Nov 25, 2023
7ff3cee
Make _WinBashStatus instances carry all their info
EliahKagan Nov 25, 2023
d5ed266
Use bytes in bash.exe check; retest no-distro case
EliahKagan Nov 25, 2023
496acaa
Handle multiple encodings for WSL error messages
EliahKagan Nov 26, 2023
d779a75
Don't assume WSL-related bash.exe error is English
EliahKagan Nov 27, 2023
9ac2438
Handle encodings better; make the sum type "public"
EliahKagan Nov 27, 2023
b07e5c7
Put back WSL on Windows CI
EliahKagan Nov 28, 2023
3303c74
Improve readability of WinBashStatus class
EliahKagan Nov 28, 2023
e00fffc
Shorten comments on _decode steps
EliahKagan Nov 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Handle multiple encodings for WSL error messages
This affects the test suite only. It improves _WinBashStatus.

When bash.exe is the WSL wrapper, and it reports an error that
prevents bash in a WSL system from ever actually being run, the
message may be encoded in a narrow or multibyte encoding, or may
instead be in Windows's native UTF-16LE encoding. This was partly
handled before, by assuming the message indicating the absence of
any installed WSL distribuions could be interpreted as UTF-8, but
matching against bytes and not actually decoding it or other error
messages. That presented a few problems, which are rememedied here:

- It turns out that this "Windows Subsystem for Linux has no
  installed distributions" message actually can be in UTF-16LE too.
  This happens when it is part of a message that also reports a
  textual error code like:

  Bash/Service/CreateInstance/GetDefaultDistro/WSL_E_DEFAULT_DISTRO_NOT_FOUND

  Therefore, narrow-encoding matching was not sufficient.

- Logged messages were hard to understand, because they were
  printing the reprs of bytes objects, which sometimes were
  UTF-16LE and thus had many "\x00" interspersed in them.

- The cases were not broken down elegantly. The ErrorWhileChecking
  case could hold a CompletedProcess, a CalledProcessError, or an
  OSError. This is now replaced with a WinError case for the rare
  scenario where CreateProcessW fails in a completely unexpected
  way, and a CheckError case for when the exit status or output of
  bash.exe indicates an error other than one we want to handle.
  CheckError has two attributes: `process` for the CompletedProcess
  (CalledProcessError is no longer used, and CompletedProcess has
  `returncode` and other attributes that provide the same info),
  and `message` for the decoded output.
  • Loading branch information
EliahKagan committed Nov 26, 2023
commit 496acaac5d2cfcbe4edf8ace1bfae452f281ff15
44 changes: 20 additions & 24 deletions test/test_index.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,13 +61,11 @@ class _WinBashStatus:
WslNoDistro = constructor()
"""Running ``bash.exe` tries to run bash on a WSL distribution, but none exists."""

ErrorWhileChecking = constructor("error_or_process")
"""Could not determine the status.
CheckError = constructor("process", "message")
"""Running ``bash.exe`` fails in an unexpected error or gives unexpected output."""

This should not trigger a skip or xfail, as it typically indicates either a fixable
problem on the test machine, such as an "Insufficient system resources exist to
complete the requested service" error starting WSL, or a bug in this detection code.
"""
WinError = constructor("exception")
"""``bash.exe`` may exist but can't run. ``CreateProcessW`` fails unexpectedly."""

@classmethod
def check(cls):
Expand All @@ -94,36 +92,34 @@ def check(cls):
if os.name != "nt":
return cls.Inapplicable()

# Use bytes because messages for different WSL errors use different encodings.
no_distro_message = b"Windows Subsystem for Linux has no installed distributions."

def error_running_bash(error):
log.error("Error running bash.exe to check WSL status: %r", error)
return cls.ErrorWhileChecking(error)
no_distro_message = "Windows Subsystem for Linux has no installed distributions."

try:
# Output rather than forwarding the test command's exit status so that if a
# failure occurs before we even get to this point, we will detect it. For
# information on ways to check for WSL, see https://superuser.com/a/1749811.
script = 'test -e /proc/sys/fs/binfmt_misc/WSLInterop; echo "$?"'
command = ["bash.exe", "-c", script]
proc = subprocess.run(command, capture_output=True, check=True)
process = subprocess.run(command, capture_output=True)
except FileNotFoundError:
return cls.Absent()
except OSError as error:
return error_running_bash(error)
except subprocess.CalledProcessError as error:
if error.returncode == 1 and error.stdout.startswith(no_distro_message):
return cls.WslNoDistro()
return error_running_bash(error)

status = proc.stdout.rstrip()
if status == b"0":
return cls.WinError(error)

encoding = "utf-16le" if b"\r\0\n\0" in process.stdout else "utf-8"
text = process.stdout.decode(encoding).rstrip() # stdout includes WSL errors.

if process.returncode == 1 and text.startswith(no_distro_message):
return cls.WslNoDistro()
if process.returncode != 0:
log.error("Error running bash.exe to check WSL status: %s", text)
return cls.CheckError(process, text)
if text == "0":
return cls.Wsl()
if status == b"1":
if text == "1":
return cls.Native()
log.error("Strange output checking WSL status: %r", proc.stdout)
return cls.ErrorWhileChecking(proc)
log.error("Strange output checking WSL status: %s", text)
return cls.CheckError(process, text)


_win_bash_status = _WinBashStatus.check()
Expand Down
Loading