Docker cross test infrastructure #297

posborne · 2016-03-07T02:14:03Z

These commits contain the first portion of new infrastructure that will allow us to build for and test on a much larger set of platforms. Nix and the tests are not compiling on many platforms right now, but I believe at this point that most of those problems are issues with nix (or libc) rather than the test infrastructure.

I want to get this out there so we can start filing issues and other people can start doing testing and development to resolve the issues popping up on various platforms. None of this is integrated in with travis at this point -- that will be phase 3. Phase 2 will be getting qemu going to run the tests for non-host platforms (this might be tricky when running in a container as /dev/kvm might need to be passed through).

This is based on the testing infrastructure in libc but modified quite a bit -- The libc infrastructure doesn't seem to be a great experience for testing locally. If this approach works out, we might want to push this approach toward libc.

The Dockerfile/Image that goes along with this is currently here:

I would have no problem moving this into the org if that makes sense down the line.

kamalmarhubi · 2016-03-07T13:25:33Z

I am really excited to look at this!

kamalmarhubi · 2016-03-09T20:51:41Z

(this might be tricky when running in a container as /dev/kvm might need to be passed through)

Could we prefer software emulation? That's what libc does for freebsd / netbsd / android on ARM. Travis can't even give us /dev/kvm if they wanted to: it's not available on GCE where they run. Or on EC2 for that matter...

kamalmarhubi · 2016-03-09T20:55:34Z

ci/run.sh

+# Tell cargo what linker to use and whatever else is required
+#
+configure_cargo() {
+  rm -rf .cargo


this seems potentially destructive

Yes. Currently, this is only executed in the container (and not in a directory that is part of a shared volume) -- May need to re-evaluate depending on how things evolve.

I guess I mean that if I accidentally run this from the top level, and I have some cargo config, I will lose it. since I keep those out of git, I would then be sad.

eg, I'd rather save the existing one, and have a trap to put the original back on exit or error

kamalmarhubi · 2016-03-09T21:04:15Z

Couple of comments so far. I'm somewhere with slow internet, and so I can't play with it because pulling the image will take a while.

posborne · 2016-03-09T22:48:31Z

(this might be tricky when running in a container as /dev/kvm might need to be passed through)

Could we prefer software emulation? That's what libc does for freebsd / netbsd / android on ARM. Travis can't even give us /dev/kvm if they wanted to: it's not available on GCE where they run. Or on EC2 for that matter...

I think we can. I am just getting around to actually testing the qemu bits and will give it a good try. I'l have some time free next week (changing day jobs and have a week off), so I should have some time to round out testing and get things ready to go for travis.

kamalmarhubi · 2016-03-11T16:54:58Z

I am just getting around to actually testing the qemu bits and will give it a good try.

👍

@posborne do you want to get this merged sooner than later?

posborne · 2016-03-12T04:37:43Z

@kamalmarhubi I am going to take one more pass at it, but then I would like to get something into master so people can start testing/fixing on various issues. Phase 2 will be figuring out how best to integrate it with CI -- We will probably need to dump travis-rust-matrix, but that has been showing some strain lately anyhow.

I'm rebuilding the docker image right now as several dependencies were missing previously (the image is quite large now as the qemu-* packages are not small).

These scripts represent a rough first cust at adding support for locally building nix and its tests for a large number of rust versions and architectures. Currently, this mostly helps with testing that nix compiles for various architectures (to the point that there are a mound of problems that need to be addressed). Once these are sorted, the code for actually running the tests for various architectures will be approached (this will require qemu and will still not target all architectures). Signed-off-by: Paul Osborne <osbpau@gmail.com>

Signed-off-by: Paul Osborne <osbpau@gmail.com>

There is now a published docker image (if a bit large) making it possible for others to start using the new test ingfrastructure. There seems to be no shortage of compile problems right now. Signed-off-by: Paul Osborne <osbpau@gmail.com>

This is handy for trying out changes to the docker image for testing things out. Signed-off-by: Paul Osborne <osbpau@gmail.com>

Signed-off-by: Paul Osborne <osbpau@gmail.com>

Later on, it will probably make sense to stop on failure -- given that most targets currently fail, however, this works a bit better for the time being. Signed-off-by: Paul Osborne <osbpau@gmail.com>

Signed-off-by: Paul Osborne <osbpau@gmail.com>

We already need to test cargo about gcc locations so it can perfom linking -- we just parse this information out to tell rust where gcc is for complation (nix-test compiles some C code for testing). Signed-off-by: Paul Osborne <osbpau@gmail.com>

There are also changes in the upstream docker images to support more targets (android on ARM, etc).

The rust-cross image is no longer all-in-one but broken out by architecture class (or platform in the case of android). Signed-off-by: Paul Osborne <osbpau@gmail.com>

This is based on libc and the new docker-based testing infrastructure via posborne/rust-cross. Signed-off-by: Paul Osborne <osbpau@gmail.com>

All of the platforms tested currently fail in some way currently, but the infrastructure itself appears to be working. As we have moved to the legacy infrastructure (required in order to use docker), the tests do run slower now. To compensate for that, i686 builds are only done on stable and we do not test every version between the current stable and the oldest version we support. Signed-off-by: Paul Osborne <osbpau@gmail.com>

posborne · 2016-03-13T21:26:02Z

Ok, things seem to be working, so I think this is ready for a review. I have the extra platforms tested set to not fail things right now until we get things sorted. As we fix things, we can remove those versions from the allow_failures set.

r? @kamalmarhubi @fiveop

fiveop · 2016-03-14T08:43:16Z

Cargo.toml

@@ -20,7 +20,7 @@ preadv_pwritev = []
 signalfd = []

 [dependencies]
-libc     = "0.2.8"
+libc     = { git = "https://github.com/rust-lang/libc.git" }
 bitflags = "0.4"


Why this change in this PR?

Ooops, I will remove this. I had this locally for testing and it slipped in.

Signed-off-by: Paul Osborne <osbpau@gmail.com>

Based on code review feedback. Signed-off-by: Paul Osborne <osbpau@gmail.com>

For platforms where the host ${CC} is appropriate, use the default implicitly rather than matching it explictly. This will allow for the llvm linker to be used if/when that happens. Signed-off-by: Paul Osborne <osbpau@gmail.com>

Signed-off-by: Paul Osborne <osbpau@gmail.com>

On the host, some undesired things could end up happening if this is executed, so we now exit and log if that is attempted. In addition, we now backup the previous cargo config if present. This will never be the case in the current build setup, but that could change if we decide to do builds in a data volume in the future. Signed-off-by: Paul Osborne <osbpau@gmail.com>

kamalmarhubi · 2016-03-15T03:46:20Z

awesome!!!

@homu r+

homu · 2016-03-15T03:46:22Z

📌 Commit d1311ff has been approved by kamalmarhubi

homu · 2016-03-15T03:46:34Z

⚡ Test exempted - status

…almarhubi Docker cross test infrastructure These commits contain the first portion of new infrastructure that will allow us to build for and test on a much larger set of platforms. Nix and the tests are *not* compiling on many platforms right now, but I believe at this point that most of those problems are issues with nix (or libc) rather than the test infrastructure. I want to get this out there so we can start filing issues and other people can start doing testing and development to resolve the issues popping up on various platforms. None of this is integrated in with travis at this point -- that will be phase 3. Phase 2 will be getting qemu going to run the tests for non-host platforms (this might be tricky when running in a container as `/dev/kvm` might need to be passed through). This is based on the testing infrastructure in libc but modified quite a bit -- The libc infrastructure doesn't seem to be a great experience for testing locally. If this approach works out, we might want to push this approach toward libc. The Dockerfile/Image that goes along with this is currently here: * https://github.com/posborne/docker-rust-cross * https://hub.docker.com/r/posborne/rust-cross/ I would have no problem moving this into the org if that makes sense down the line.

posborne force-pushed the docker-cross-test-infrastructure branch from c99f5c3 to 65ede19 Compare March 7, 2016 05:46

posborne mentioned this pull request Mar 7, 2016

Add support for MIPS targets #301

Merged

kamalmarhubi reviewed Mar 9, 2016
View reviewed changes

kamalmarhubi mentioned this pull request Mar 13, 2016

android: Fix reference to libc::ioctl #312

Merged

posborne added 11 commits March 13, 2016 00:41

testing: configure linker via .cargo/config

6ffe725

Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: add documentation covering the basics

b5a9227

Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: switch to posborne/rust-cross

22b1820

There is now a published docker image (if a bit large) making it possible for others to start using the new test ingfrastructure. There seems to be no shortage of compile problems right now. Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: allow the docker image to be overriden

3f2b70a

This is handy for trying out changes to the docker image for testing things out. Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: remove set -x from ci/run.sh

806e0a7

Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: disable some targets and continue running on failure

d95f4d5

Later on, it will probably make sense to stop on failure -- given that most targets currently fail, however, this works a bit better for the time being. Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: fix test exeuction paths with move to docker container

b5b7042

Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: disable thread parallelization in docker testing

5ddd8b2

Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: fix gcc/linker paths for several targets

b69558c

There are also changes in the upstream docker images to support more targets (android on ARM, etc).

posborne force-pushed the docker-cross-test-infrastructure branch from bdefc80 to b69558c Compare March 13, 2016 06:41

testing: move to new posborne/rust-cross images

cd3fdaf

The rust-cross image is no longer all-in-one but broken out by architecture class (or platform in the case of android). Signed-off-by: Paul Osborne <osbpau@gmail.com>

posborne force-pushed the docker-cross-test-infrastructure branch from b338ccc to f9371fc Compare March 13, 2016 20:15

testing: first shot at testing with travis for arm, mips, android, ...

6ad81d1

This is based on libc and the new docker-based testing infrastructure via posborne/rust-cross. Signed-off-by: Paul Osborne <osbpau@gmail.com>

posborne force-pushed the docker-cross-test-infrastructure branch from f9371fc to 6ad81d1 Compare March 13, 2016 20:16

posborne force-pushed the docker-cross-test-infrastructure branch from 3d7bac4 to 3b7b15d Compare March 13, 2016 21:23

fiveop reviewed Mar 14, 2016
View reviewed changes

posborne added 6 commits March 14, 2016 12:45

libc: revert change to libc version

ef8f643

Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: move echo statement within docker run script

02af925

Based on code review feedback. Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: use default linker for appropriate platforms

5eb1fa6

For platforms where the host ${CC} is appropriate, use the default implicitly rather than matching it explictly. This will allow for the llvm linker to be used if/when that happens. Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: update documentation related to testing

4e245f4

Signed-off-by: Paul Osborne <osbpau@gmail.com>

testing: explicitly check TRAVIS_OS_NAME is osx

72f6754

Signed-off-by: Paul Osborne <osbpau@gmail.com>

posborne force-pushed the docker-cross-test-infrastructure branch from 0ed9167 to d1311ff Compare March 14, 2016 19:34

homu merged commit d1311ff into nix-rust:master Mar 15, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker cross test infrastructure #297

Docker cross test infrastructure #297

posborne commented Mar 7, 2016

kamalmarhubi commented Mar 7, 2016

kamalmarhubi commented Mar 9, 2016

kamalmarhubi Mar 9, 2016

posborne Mar 9, 2016

kamalmarhubi Mar 11, 2016

kamalmarhubi Mar 11, 2016

kamalmarhubi commented Mar 9, 2016

posborne commented Mar 9, 2016

kamalmarhubi commented Mar 11, 2016

posborne commented Mar 12, 2016

posborne commented Mar 13, 2016

fiveop Mar 14, 2016

posborne Mar 14, 2016

kamalmarhubi commented Mar 15, 2016

homu commented Mar 15, 2016

homu commented Mar 15, 2016

Docker cross test infrastructure #297

Docker cross test infrastructure #297

Conversation

posborne commented Mar 7, 2016

kamalmarhubi commented Mar 7, 2016

kamalmarhubi commented Mar 9, 2016

kamalmarhubi Mar 9, 2016

Choose a reason for hiding this comment

posborne Mar 9, 2016

Choose a reason for hiding this comment

kamalmarhubi Mar 11, 2016

Choose a reason for hiding this comment

kamalmarhubi Mar 11, 2016

Choose a reason for hiding this comment

kamalmarhubi commented Mar 9, 2016

posborne commented Mar 9, 2016

kamalmarhubi commented Mar 11, 2016

posborne commented Mar 12, 2016

posborne commented Mar 13, 2016

fiveop Mar 14, 2016

Choose a reason for hiding this comment

posborne Mar 14, 2016

Choose a reason for hiding this comment

kamalmarhubi commented Mar 15, 2016

homu commented Mar 15, 2016

homu commented Mar 15, 2016