Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Linux and macOS with GitHub Actions #4081

Merged
merged 7 commits into from
Oct 12, 2019

Conversation

hugovk
Copy link
Member

@hugovk hugovk commented Sep 22, 2019

For #3606.

Changes proposed in this pull request:

  • This does the same tests as the "native" (ie. non-Docker) builds on Travis CI:

    • pypy3
    • pypy2
    • 3.7
    • 3.6 PYTHONOPTIMIZE=1
    • 3.5 PYTHONOPTIMIZE=2
    • 2.7
  • Not available on GitHub Actions:

    • "2.7_with_system_site_packages" # For PyQt4
    • 3.8-dev
  • Also rename some variables in lint.yml for consistency

Travis scripts

Happily GitHub Actions runs the .travis/*.sh scripts with no problems. There's an if [ "$TRAVIS_PYTHON_VERSION" == "2.7" ]; then make doccheck; fi in one, I've added the same doccheck as its own step here for GHA.

Linux

This only runs on ubuntu-latest, currently ubuntu-18.04 Bionic. ubuntu-16.04 Xenial is also available.

Coverage

This does coverage.

TODO

Now, unfortunately these tokens are not yet available for forks. Travis and AppVeyor don't use tokens because Codecov can autodetect those. That's not yet the case for Azure Pipelines and GitHub Actions. There's a workaround for AZP to make secrets available to forks. I've requested it for GHA and they said they'll take a look.

Downsides of this:

  • It's not ready yet

  • It would be possible for people to find out these tokens, but as Hynek mentions in his AZP workaround: "However, the token is worthless for anything except uploading coverage and it’s easy to see when someone does it."

  • It would make other secrets available to forks (eg. GitHub Action to add issue or PR to project #4071)

  • Alternatively, how about we just include them as plaintext in the YAML?

Downsides:

This was referenced Sep 22, 2019
@hugovk hugovk changed the title Test with GitHub Actions Test Linux with GitHub Actions Sep 22, 2019
@hugovk hugovk added the Linux label Sep 22, 2019
@hugovk
Copy link
Member Author

hugovk commented Sep 22, 2019

Updated to add macOS into the matrix! 🍏

It tests the same Python versions, and just needed a slightly different install step.

@hugovk hugovk added the macOS label Sep 22, 2019
@hugovk hugovk changed the title Test Linux with GitHub Actions Test Linux and macOS with GitHub Actions Sep 22, 2019
.github/workflows/test.yml Outdated Show resolved Hide resolved
@nulano nulano mentioned this pull request Sep 23, 2019
19 tasks
@hugovk
Copy link
Member Author

hugovk commented Sep 24, 2019

@python-pillow/pillow-team Any objections to including coverage tokens as plaintext? Will also unblock the Actions/Windows build too: #4084.

@aclark4life
Copy link
Member

@hugovk Assuming no known security issues in doing that, no objections I can think of

@hugovk hugovk force-pushed the test-with-github-actions branch 2 times, most recently from 8c4c42a to 5879918 Compare September 26, 2019 11:07
@hugovk
Copy link
Member Author

hugovk commented Sep 26, 2019

Actions: One of the jobs timed out, so I clicked to "Re-run all checks" and all the checkouts failed. Reported here.

@hugovk
Copy link
Member Author

hugovk commented Sep 27, 2019

Rebased on master.

@radarhere
Copy link
Member

Is the fact that checkouts might fail on a rerun enough of a problem to keep this from being merged at the moment, or is it something we can live with?

@hugovk
Copy link
Member Author

hugovk commented Oct 6, 2019

It's enough to stop GHA fully replacing Travis CI etc., but as @nulano said in #4084 (comment), we should keep the old CIs at least until GHA is out of beta (13 November).

There's also some question about coverage (#4084 (comment)), but I think we're good to merge, we can see how it goes and iterate.

@hugovk
Copy link
Member Author

hugovk commented Oct 8, 2019

Okay to merge?

@hugovk hugovk force-pushed the test-with-github-actions branch 2 times, most recently from 367ce3d to 707c991 Compare October 12, 2019 09:14
@hugovk
Copy link
Member Author

hugovk commented Oct 12, 2019

I rebased and had a go installing PyQt5 like this:

index d6676ace..3834da90 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -38,6 +38,8 @@ jobs:
       if: startsWith(matrix.os, 'ubuntu')
       run: |
         .travis/install.sh
+        sudo apt-get -qq install pyqt5-dev-tools
+        pip install pyqt5

     - name: Install macOS dependencies
       if: startsWith(matrix.os, 'macOS')

But it failed like this:

2019-10-12T08:19:21.3815706Z Tests/test_psdraw.py::TestPsDraw::test_stdout PASSED                     [ 98%]
2019-10-12T08:19:21.6143737Z Tests/test_pyroma.py::TestPyroma::test_pyroma PASSED                     [ 98%]
2019-10-12T08:19:21.6203852Z Fatal Python error: Aborted
2019-10-12T08:19:21.6204200Z 
2019-10-12T08:19:21.6204501Z Current thread 0x00007fdb4d090740 (most recent call first):
2019-10-12T08:19:21.6205356Z   File "/home/runner/work/Pillow/Pillow/Tests/test_imageqt.py", line 37 in setUp
2019-10-12T08:19:21.6206011Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/unittest/case.py", line 624 in run
2019-10-12T08:19:21.6206431Z   File "/home/runner/work/Pillow/Pillow/Tests/helper.py", line 63 in run
2019-10-12T08:19:21.6206883Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/unittest/case.py", line 676 in __call__
2019-10-12T08:19:21.6208127Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/unittest.py", line 207 in runtest
2019-10-12T08:19:21.6231329Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/runner.py", line 125 in pytest_runtest_call
2019-10-12T08:19:21.6232522Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
2019-10-12T08:19:21.6233090Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/manager.py", line 86 in <lambda>
2019-10-12T08:19:21.6233627Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
2019-10-12T08:19:21.6234165Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
2019-10-12T08:19:21.6234702Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/runner.py", line 201 in <lambda>
2019-10-12T08:19:21.6235241Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/runner.py", line 229 in from_call
2019-10-12T08:19:21.6236041Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/runner.py", line 201 in call_runtest_hook
2019-10-12T08:19:21.6236775Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/runner.py", line 176 in call_and_report
2019-10-12T08:19:21.6237326Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/runner.py", line 95 in runtestprotocol
2019-10-12T08:19:21.6237909Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/runner.py", line 80 in pytest_runtest_protocol
2019-10-12T08:19:21.6238458Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
2019-10-12T08:19:21.6238990Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/manager.py", line 86 in <lambda>
2019-10-12T08:19:21.6239519Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
2019-10-12T08:19:21.6240053Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
2019-10-12T08:19:21.6240616Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/main.py", line 256 in pytest_runtestloop
2019-10-12T08:19:21.6241158Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
2019-10-12T08:19:21.6241699Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/manager.py", line 86 in <lambda>
2019-10-12T08:19:21.6242613Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
2019-10-12T08:19:21.6243160Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
2019-10-12T08:19:21.6243689Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/main.py", line 235 in _main
2019-10-12T08:19:21.6244210Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/main.py", line 191 in wrap_session
2019-10-12T08:19:21.6244765Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/main.py", line 228 in pytest_cmdline_main
2019-10-12T08:19:21.6245316Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
2019-10-12T08:19:21.6245948Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/manager.py", line 86 in <lambda>
2019-10-12T08:19:21.6246497Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
2019-10-12T08:19:21.6247029Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
2019-10-12T08:19:21.6247568Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/_pytest/config/__init__.py", line 90 in main
2019-10-12T08:19:21.6248088Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/site-packages/pytest.py", line 101 in <module>
2019-10-12T08:19:21.6248343Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/runpy.py", line 85 in _run_code
2019-10-12T08:19:21.6248612Z   File "/opt/hostedtoolcache/Python/3.7.4/x64/lib/python3.7/runpy.py", line 193 in _run_module_as_main
2019-10-12T08:19:40.5533965Z .travis/test.sh: line 5: 63421 Aborted                 (core dumped) python -m pytest -v -x --cov PIL --cov-report term Tests
2019-10-12T08:19:40.5534978Z Tests/test_qt_image_fromqpixmap.py::TestFromQPixmap::test_sanity 
2019-10-12T08:19:40.5572985Z ##[error]Process completed with exit code 134.
2019-10-12T08:19:40.5729825Z Cleaning up orphan processes

https://github.com/python-pillow/Pillow/commit/367ce3d370f82e0c5c0047869be444b99b7685f7/checks?check_suite_id=262280147

So took that out for now. Also added fail-fast: false, like we have on Windows GHA, so that a single failed job does not cancel the others.

Will merge when green.

@hugovk hugovk merged commit 6595ce1 into python-pillow:master Oct 12, 2019
@hugovk hugovk deleted the test-with-github-actions branch October 12, 2019 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants