Python 3.7 recently broken (no _bz2 module) on macOS 13

[gsnedders]

Description

Recently (i.e., since last week), we've started seeing a number of Python 3.7 tasks failing with:

../manifest/download.py:2: in <module>
    import bz2
/Users/runner/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/bz2.py:19: in <module>
    from _bz2 import BZ2Compressor, BZ2Decompressor
E   ModuleNotFoundError: No module named '_bz2'

Presumably something in the new image caused CPython to fail to compile the _bz2 module?

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• macOS 11
• macOS 12
• macOS 13
• Windows Server 2019
• Windows Server 2022

Image version and build link

20230611.2

https://dev.azure.com/web-platform-tests/wpt/_build/results?buildId=102338&view=logs&jobId=cd4b0752-91ac-5ee3-1416-93dd445ec171&j=5eb1b5a9-6d44-5065-06d8-b95593136516&t=2324b04f-e597-4527-9bcd-b3f55a7684b6

Is it regression?

20230524.1

Expected behavior

The Python 3.7 _bz2 module to be installed.

Actual behavior

The Python 3.7 _bz2 module is not installed.

Repro steps

python3.7 -c 'import bz2'

[mikhailkoliada]

Transferring to setup-python as they are responsible for tool cached python

CC @dmitry-shibanov @MaksimZhukov

[jcscottiii]

I wanted to provide an additional data point to help with the investigation.

It seems that it is a regression in the latest patch version of 3.7 installed by the Mac agents.

We recognized that our builds last succeeded when python 3.7.16 was installed. Because we had only major.minor pinned, the Mac agents installed python 3.7.17. And that was when the builds started to fail.

As a temporary workaround, we have pinned the Python version to 3.7.16 to make the builds work again.

So you can look into how Python 3.7.17 was built. My hunch would be that the latest python install script does not have libbz2-dev installed prior to building it.

[MaksimZhukov]

Hello @gsnedders! Thank you for reporting this!
We will investigate the issue!

[seanlaw]

Thank you for your response @parafoxia. I'll keep the issue opened until the hosted runners are updated.

@dmitry-shibanov Any idea when the hosted runners might be updated and available?

[dmitry-shibanov]

Hello @seanlaw. I think it should be uploaded at the end of this week.

[seanlaw]

Hello everyone. We uploaded a new version of python 3.7.17 with the fix. Could you please try to use this snippet to confirm that it works as expected?

      - run: rm -rf $RUNNER_TOOL_CACHE/Python/3.7.17
      - uses: actions/setup-python@v4
        with:
          python-version: 3.7.17

@dmitry-shibanov I would also like to confirm that this worked for me as well.

[macanudo527]

@dmitry-shibanov It seems like the hosted runners have been updated. Tests are passing without the added snippet.

[dmitry-shibanov]

Hello everyone. Sorry for the late response. The python 3.7.17 was updated in the hosted toolcache directory. For now I'm going to close the issue.