Support for unprivileged docker daemon

[vorner]

I have my dockerd configured to run under an unprivileged user. However, it seems this is not really supported by cross:

$ cross test --target armv7-unknown-linux-musleabihf
docker: Error response from daemon: Privileged mode is incompatible with user namespaces.
See 'docker run --help'.
error: `"docker" "run" "--privileged" "--rm" "-it" "ubuntu:16.04" "sh" "-c" "apt-get update && apt-get install --no-install-recommends -y binfmt-support qemu-user-static"` failed with exit code: Some(125)
note: run with `RUST_BACKTRACE=1` for a backtrace

It fails with a different error with build, which probably could be solved by granting some unreasonable permissive permissions to ~/.cargo:

docker: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"rootfs_linux.go:54: mounting \\\"/home/vorner/.cargo\\\" to rootfs \\\"/var/lib/docker/231072.231072/btrfs/subvolumes/9a8dd48eec320f381e259e8640fb3414aa0baa4874c540c1334396d7a739bffc\\\" at \\\"/cargo\\\" caused \\\"stat /home/vorner/.cargo: permission denied\\\"\"".

Would it be viable to support these more paranoid modes of operation, or at least document this is not supported and why?

[japaric]

Could you try the master branch? (cargo install --git https://github.com/japaric/cross). I think someone already fixed this in #90 / #96 but I haven't made a release with those changes.

[vorner]

It definitely helps. Build works. But test still doesn't work, I think it is related to this:

Setting up binfmt-support (2.1.6-1) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up qemu-user-static (1:2.5+dfsg-5ubuntu10.14) ...
mount: only root can use "--options" option (effective UID is 231072)
update-binfmts: warning: Couldn't mount the binfmt_misc filesystem on /proc/sys/fs/binfmt_misc.

Then it complains that:

     Running /target/armv7-unknown-linux-musleabihf/debug/deps/aggregator-9187e04453fe8de2
error: could not execute process `/target/armv7-unknown-linux-musleabihf/debug/deps/aggregator-9187e04453fe8de2` (never executed)

I guess getting away without setting up the binfmt_misc won't be easy?

[japaric]

In principle we can set up the binfmt_misc stuff outside docker then we wouldn't need that privileged docker call but I never got around to implement that. Setting binfmt_misc outside docker would still require root privilege though.

[japaric]

Alternatively now that target.$T.runner is a thing we could set that to QEMU to have Cargo call test binaries using QEMU instead of relying on binfmt_misc. Those Cargo runners are not in stable so such change would be breaking and would only be usable from nightly at least until the feature rides the train and reaches stable.

[vorner]

Maybe as an option? Or detect if it is running nightly?

[japaric]

@vorner PR #122, which has not yet landed, has the changes I mentioned above, if you want to try it.

[vorner]

Hmm. Am I using it in a wrong way? What should I look for? The error message seems a bit unhelpful.

$ cross test --target armv7-unknown-linux-musleabihf --verbose
+ "rustup" "target" "list"
+ "cargo" "fetch" "--manifest-path" "/home/vorner/prog/cross/Cargo.toml"
+ "rustc" "--print" "sysroot"
+ "docker" "run" "--userns" "host" "--rm" "--user" "1000:407" "-e" "CARGO_HOME=/cargo" "-e" "CARGO_TARGET_DIR=/target" "-e" "USER=vorner" "-e" "XARGO_HOME=/xargo" "-v" "/home/vorner/.xargo:/xargo" "-v" "/home/vorner/.cargo:/cargo" "-v" "/home/vorner/prog/cross:/project:ro" "-v" "/home/vorner/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu:/rust:ro" "-v" "/home/vorner/prog/cross/target:/target" "-w" "/project" "-it" "japaric/armv7-unknown-linux-musleabihf:latest" "sh" "-c" "PATH=$PATH:/rust/bin \"cargo\" \"test\" \"--target\" \"armv7-unknown-linux-musleabihf\" \"--verbose\""
       Fresh rustc-demangle v0.1.3
       Fresh cfg-if v0.1.0
       Fresh libc v0.2.18
       Fresh semver-parser v0.7.0
       Fresh lazy_static v0.2.8
       Fresh gcc v0.3.45
       Fresh semver v0.1.20
       Fresh winapi-build v0.1.1
       Fresh winapi v0.2.8
       Fresh rustc-serialize v0.3.22
       Fresh semver v0.6.0
       Fresh rustc_version v0.1.7
       Fresh toml v0.2.1
       Fresh backtrace-sys v0.1.5
       Fresh dbghelp-sys v0.2.0
       Fresh kernel32-sys v0.2.2
       Fresh backtrace v0.3.0
       Fresh error-chain v0.7.2
       Fresh cross v0.1.12-dev (file:///project)
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `/target/armv7-unknown-linux-musleabihf/debug/deps/cross-4c8a6115151a421e`
error: could not execute process `/target/armv7-unknown-linux-musleabihf/debug/deps/cross-4c8a6115151a421e` (never executed)

(this is an attempt on cross itself)

[japaric]

@vorner hmm, that's probably because you are using a "-dev" version (see cross -V). On "-dev" releases cross will use local Docker images so you would need to build a Docker image locally, with the tag "v0.1.12-dev", to make this work.

In any case the PR I linked above has been r+-ed and I'll release a new (stable) Cross version when it lands.

[vorner]

Hello. Sorry for taking so long to get back to this. Yes, it works now, tested successfully few days ago.

[jamesmunns]

Closing as @vorner says it works.