Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add integration / unit tests to github actions #2690

Merged
merged 1 commit into from
Nov 30, 2020

Conversation

kolyshkin
Copy link
Contributor

@kolyshkin kolyshkin commented Nov 24, 2020

This is Ubuntu 20.04 so alas cgroup v1 testing only.

The upside is it's pretty fast ⏩ we have results of unit + integration tests
for both go 1.14 and 1.15 within 5 minutes. This seems useful now when travis-ci.org
is throttling jobs 😞

rootless tests take longer time so we run them in parallel
with non-rootless ones. Also, to balance the time, we run
unit tests when rootless is not set. IOW, 'rootless' in job
name means it's only rootless integration tests, and no rootless
means it's unit tests, integration tests with fs driver, and
integration tests with systemd driver.

Note, script is used to run integration tests to provide a tty
(which by default is not available on gh actions).

@kolyshkin kolyshkin force-pushed the gh-act-int branch 5 times, most recently from dbb8301 to 2f0dec8 Compare November 24, 2020 23:21
@kolyshkin kolyshkin force-pushed the gh-act-int branch 9 times, most recently from 9590745 to d10a4e3 Compare November 27, 2020 11:38
@kolyshkin
Copy link
Contributor Author

OK, half the tests are working now, the other half complains:

time="2020-11-27T11:40:27Z" level=error msg="open /dev/tty: no such device or address"

One way to fix is to use run tests under docker.

There's a bug with lots of comments about it: actions/runner#241

Will take a look later.

@kolyshkin kolyshkin marked this pull request as draft November 27, 2020 11:57
@kolyshkin kolyshkin force-pushed the gh-act-int branch 8 times, most recently from ee5f6b9 to 28899fc Compare November 28, 2020 02:05
@kolyshkin kolyshkin changed the title [WIP] ci: add integration tests to github actions [WIP] ci: add integration / unit tests to github actions Nov 28, 2020
@kolyshkin kolyshkin marked this pull request as ready for review November 28, 2020 02:45
@kolyshkin kolyshkin force-pushed the gh-act-int branch 2 times, most recently from 8354ffe to 2943f07 Compare November 28, 2020 02:48
@kolyshkin kolyshkin force-pushed the gh-act-int branch 2 times, most recently from d1b6288 to ee5ca7d Compare November 28, 2020 02:53
@kolyshkin
Copy link
Contributor Author

OK I think this is ready for prime time. I don't mean to replace travis (yet?), it's just in addition to get a faster test results.

@AkihiroSuda @mrunalp PTAL

This is Ubuntu 20.04 so cgroup v1 testing only :(
The upside is, it's pretty fast (currently all the tests finish
within 5 minutes).

rootless tests take longer time so we run them in parallel
with non-rootless ones. Also, to balance the time, we run
unit tests when rootless is not set. IOW, 'rootless' in job
name means it's only rootless integration tests, and no rootless
means it's unit tests, integration tests with fs driver, and
integration tests with systemd driver.

Note, `script` is used to run integration tests to provide a tty
(which by default is not available on gh actions).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
@kolyshkin kolyshkin changed the title [WIP] ci: add integration / unit tests to github actions ci: add integration / unit tests to github actions Nov 28, 2020
@kolyshkin
Copy link
Contributor Author

Ahh, interesting...

not ok 23 global --debug to --log
# (in test file tests/integration/debug.bats, line 41)
#   `[[ "${output}" == *"child process in init()"* ]]' failed
# runc spec --rootless (status=0):

@kolyshkin
Copy link
Contributor Author

Ahh, interesting...

not ok 23 global --debug to --log
# (in test file tests/integration/debug.bats, line 41)
#   `[[ "${output}" == *"child process in init()"* ]]' failed
# runc spec --rootless (status=0):

I barely remember something about this logging being a tad racy, and I've been trying to fix that in #2487.

Oh yes, this ugly commit: f9b4cf0

Alas, I haven't finished that PR and I don't even remember why now. Let me try to revive it...

@kolyshkin
Copy link
Contributor Author

Let me try to revive it...

In the meantime, this PR is fine, the flake in debug logging is known problem (and a very minor one, since this is runc debug logging no one but a few runc devs care about).

@kolyshkin
Copy link
Contributor Author

@AkihiroSuda @mrunalp PTAL

@AkihiroSuda
Copy link
Member

FYI previously there was a decision not to migrate to GHA (#2512), but I guess the context has changed after Travis changed the free(-as-in-beer) plan. So LGTM.

cc @cyphar WDYT?

@kolyshkin
Copy link
Contributor Author

FYI previously there was a decision not to migrate to GHA (#2512), but I guess the context has changed after Travis changed the free(-as-in-beer) plan. So LGTM.

Sorry I overlooked it. Indeed, the situation with Travis is unclear, and GH actions are (currently) both fast and free (and in case they won't be, we could ask LF to pay for it).

I am also thinking about using https://cirrus-ci.org/ as it supports Fedora so we can test cgroupv2.

@mrunalp mrunalp merged commit 2b92c25 into opencontainers:master Nov 30, 2020
@AkihiroSuda
Copy link
Member

I am also thinking about using https://cirrus-ci.org/ as it supports Fedora so we can test cgroupv2.

Seems to need nested VM? (which is also supported by GHA on macOS instances)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants