Value too large for defined data type when cargo build on armv7 with qemu

[jdrouet]

Problem

Trying to build an app for armv7 in a docker container with qemu.
The current result is explained here with the steps to reproduce it : https://github.com/jdrouet/docker-bug-value-too-long

Steps

1. install docker
2. install docker-buildx
3. follow the steps here

Possible Solution(s)
No idea.

Notes
The guys from docker claim it's coming from cargo.

Output of cargo version:

The one from the docker image rust:1-slim-buster

[ehuss]

Thanks for the report. I think this is a duplicate of #7451. There is some sort of strange interaction of Linux Large File Support with a 64-bit host and a 32-bit docker image running qemu. There's some more discussion on that issue, but in particular see https://bugs.launchpad.net/qemu/+bug/1805913. I'm not sure there is anything Cargo can do about this. I suspect the underlying problem is that libgit2 uses size-agnostic functions like readdir instead of explicitly using readdir64. It may be possible to build libgit2 with -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 (or getconf LFS_CFLAGS), but I have no idea if that will work or what the consequences would be (it seems like it could have portability issues). This may also just be an issue with qemu.

[diabloxenon]

Hello all,

I have a similar use case i.e. (Qemu userspace emulation for armv7 on x86_64 arch). After chrooting inside the image, i tried building my project repository using that and this was the output.

[root@mnt ~]# cargo build --target=armv7-unknown-linux-gnueabihf
    Updating crates.io index
warning: spurious network error (2 tries remaining): could not read directory '/root/.cargo/registry/index/github.com-1285ae84e5963aae/.git//refs': Value too large for defined data type; class=Os (2)
warning: spurious network error (1 tries remaining): could not read directory '/root/.cargo/registry/index/github.com-1285ae84e5963aae/.git//refs': Value too large for defined data type; class=Os (2)
error: failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  could not read directory '/root/.cargo/registry/index/github.com-1285ae84e5963aae/.git//refs': Value too large for defined data type; class=Os (2)
[root@mnt ~]# 

However, when fetching it manually using curl https://github.com/rust-lang/crates.io-index from chroot, this worked exactly as it should have. Because of that, I also tested this on qemu aarch64 image and there it worked perfectly.

As suggested earlier by @ehuss, it seems the issue is still persistent to this date with qemu emulation of armv7. The cargo binaries are definitely not the culprit here, and more likely libgit2 seems to be causing this issue.

Internally, there is some incompatiblity going on, even after following the suggestion for building the library with parameters provided by getconf LFS_CFLAGS, the issue remains the same.

Lastly, for this quick and simple test case, I would prefer to use command cargo install exa inside an emulated armv7 image already installled with Rust, for any future testing regarding to its compatiblity.

EDIT: I also proposed a simple solution for anyone that is currenly facing this problem, and until its resolution. #9545

[tillsteinbach]

If someone is coming here and also needs a workaround for a docker build:
Put this in the beginning of the Dockerfile:
# syntax = docker/dockerfile:experimental
Then do something like this:
RUN --security=insecure mkdir -p /root/.cargo && chmod 777 /root/.cargo && mount -t tmpfs none /root/.cargo && some_command_that_uses_cargo