You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
*`multiarch/qemu-user-static` image container includes both a register script to register binfmt_misc entries and all the `/usr/bin/qemu-$arch-static` binary files in the container in it.
80
+
*`multiarch/qemu-user-static:$to_arch` images are aliases of `multiarch/qemu-user-static:x86_64-$to_arch`. A `multiarch/qemu-user-static:$to_arch` images only includes the `$to_arch`'s `/usr/bin/qemu-$to_arch-static` binary file in it.
81
+
*`multiarch/qemu-user-static:register` image has only the register script binfmt_misc entries.
82
+
83
+
`multiarch/qemu-user-static` and `multiarch/qemu-user-static:register` images execute the register script that registers below kind of `/proc/sys/fs/binfmt_misc/qemu-$arch` files for all supported processors except the current one in it when running the container. See binfmt_misc manual [2] for detail of the files.
84
+
As the `/proc/sys/fs/binfmt_misc` are common between host and inside of container, the register script modifies the file on host.
85
+
86
+
```
87
+
$ cat /proc/sys/fs/binfmt_misc/qemu-$arch
88
+
enabled
89
+
interpreter /usr/bin/qemu-$arch-static
90
+
flags: F
91
+
offset 0
92
+
magic 7f454c460201010000000000000000000200b700
93
+
mask ffffffffffffff00fffffffffffffffffeffffff
94
+
```
95
+
96
+
The `--reset` option is implemented at the register script that executes `find /proc/sys/fs/binfmt_misc -type f -name 'qemu-*' -exec sh -c 'echo -1 > {}' \;` to remove binfmt_misc entry files before register the entry.
97
+
When same name's file `/proc/sys/fs/binfmt_misc/qemu-$arch` exists, the register command is failed with an error message "sh: write error: File exists".
98
+
99
+
```
100
+
$ docker run --rm --privileged multiarch/qemu-user-static [--reset][--help][-p yes][options]
101
+
```
102
+
103
+
On below image, we can not specify `-p yes` (`--persistent yes`) option. Because an interpreter's existance is checked when registering a binfmt_misc entry. As the interpreter does not exist in the container, the register script finshes with the error.
104
+
105
+
```
106
+
$ docker run --rm --privileged multiarch/qemu-user-static:register [--reset][--help][options]
107
+
```
108
+
109
+
Then the register script executes QEMU's [scripts/qemu-binfmt-conf.sh](https://github.com/qemu/qemu/blob/master/scripts/qemu-binfmt-conf.sh) script with options.
110
+
You can check `usage()` in the file about the options.
$ docker run --rm -t "test/integration/ubuntu" uname -m
171
+
aarch64
172
+
```
173
+
174
+
If you have `qemu-$arch-static` binary files on your local environment, you can set it to the container by `docker -v` volume mounted file.
175
+
176
+
```
177
+
$ docker run --rm --privileged multiarch/qemu-user-static:register --reset
178
+
179
+
$ docker run --rm -t arm64v8/ubuntu uname -m
180
+
standard_init_linux.go:211: exec user process caused "no such file or directory"
181
+
182
+
$ docker run --rm -t -v /usr/bin/qemu-aarch64-static:/usr/bin/qemu-aarch64-static arm64v8/ubuntu uname -m
183
+
aarch64
184
+
```
185
+
186
+
### multiarch compatible images [DEPRECATED]
187
+
188
+
The concept of "compatible images" are deprecated because **multiarch/qemu-user-static** can build and run standard multi-architecture container images without the multiarch compatible images now. But you can refer the document [Compatible images](docs/compatible_images.md).
189
+
190
+
The compatible image is the one to add `/usr/bin/qemu-$arch-static` binary inside of the container based on the standard arch specific container.
191
+
Last time, we could not register binfmt_misc entry with `flags: F` (persistent option).
192
+
When `flags: F` was not set, the interpreter always needed to be existed inside of the container to run the arch container.
193
+
194
+
See [Users guide](docs/users_guide.md) for detail.
195
+
196
+
## Contributing
197
+
198
+
We encourage you to contribute to **multiarch/qemu-user-static**! Please check out the [Contributing to multiarch/qemu-user-static guide](docs/CONTRIBUTING.md) for guidelines about how to proceed.
199
+
200
+
See [Developers guide](docs/developers_guide.md) for detail.
21
201
22
202
## Supported host architectures
23
203
@@ -28,32 +208,14 @@ Run `uname -m` to check it on your environment.
We provide information to understand this system in this document.
4
+
5
+
## Technology overview
6
+
7
+
### qemu-user-static
8
+
9
+
qemu-user-static is a collection of `qemu-$arch-static` "static" binary files that emulates application process (QEMU "user" mode) and binfmt_misc related files. [1]
10
+
In this system, Fedora project's qemu-user-static RPM is used as the input data. qemu-user-static RPM is sub package of qemu RPM. [2]
11
+
12
+
`qemu-$arch-static` file is just an interpreter to run the archtecture specfic binary. Below is an example to run aarch64 specifc binary `bin/hello-aarch64` on `qemu-aarch64-static`.
13
+
14
+
```
15
+
$ uname -m
16
+
x86_64
17
+
18
+
$ file bin/hello-aarch64
19
+
bin/hello-aarch64: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=fa19c63e3c60463e686564eeeb0937959bd6f559, for GNU/Linux 3.7.0, not stripped, too many notes (256)
20
+
21
+
$ bin/hello-aarch64
22
+
bash: bin/hello-aarch64: cannot execute binary file: Exec format error
23
+
24
+
$ qemu-aarch64-static bin/hello-aarch64
25
+
Hello World!
26
+
```
27
+
28
+
`qemu-$cpu-static` can run the architecture specific binary.
29
+
30
+
31
+
### qemu-user-static and binfmt_misc
32
+
33
+
qemu-user-static becomes powerful when it is used with binfmt_misc [3].
34
+
If you are interested in the C language level manual, see [4].
35
+
36
+
Here is an example of the most typical and recommended way to run different architecture's container.
37
+
In this example, aarch64 (ARM 64-bit) container are executed on host architecture x86_64.
38
+
The example shows what the container image is doing.
39
+
40
+
```
41
+
$ uname -m
42
+
x86_64
43
+
44
+
$ ls /proc/sys/fs/binfmt_misc/qemu-aarch64
45
+
ls: cannot access '/proc/sys/fs/binfmt_misc/qemu-aarch64': No such file or directory
46
+
47
+
$ docker run --rm -t arm64v8/ubuntu uname -m
48
+
standard_init_linux.go:211: exec user process caused "exec format error"
49
+
50
+
$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
51
+
52
+
$ ls /proc/sys/fs/binfmt_misc/qemu-aarch64
53
+
/proc/sys/fs/binfmt_misc/qemu-aarch64
54
+
55
+
$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64
56
+
enabled
57
+
interpreter /usr/bin/qemu-aarch64-static
58
+
flags: F
59
+
offset 0
60
+
magic 7f454c460201010000000000000000000200b700
61
+
mask ffffffffffffff00fffffffffffffffffeffffff
62
+
63
+
$ docker run --rm -t arm64v8/ubuntu uname -m
64
+
aarch64
65
+
```
66
+
67
+
In this system, understanding the difference of the behavior of 2 patterns of flags: `flags: ` (empty flag) and `flags: F` is important.
68
+
According to [3], the actual operation and behavior are as follows.
69
+
70
+
*`# echo ":$name:$type:$offset:$magic:$mask:$interpreter:$flags" > /proc/sys/fs/binfmt_misc/register` to add a binary format entry.
71
+
*`# echo -1 > /proc/sys/fs/binfmt_misc/qemu-$arch` to remove a qemu binary format entry.
72
+
* If the entry file's `flags` is empty, the exsistance of the interpreter is checked at run time.
73
+
* If the entry file's `flags` is `flags: F`, the existance of the interpreter is checked when registering the entry.
74
+
75
+
### qemu-user-static, binfmt_misc and container
76
+
77
+
A point to keep in mind when using qemu-user-static and binfmt_misc in container, is binfmt_misc `/proc/sys/fs/binfmt_misc` files `register`, `status` and `qemu-$arch` are shared and commonly used between host and inside of container. As a result, a script executed in container can modify `/proc/sys/fs/binfmt_misc` files on host OS.
78
+
binfmt_misc is a feature of kernel. A container uses the host OS's kernel.
79
+
80
+
## Programs input & output
81
+
82
+
In this section, we describe a program's input and output.
83
+
This repository is a pipeline system by using Travis CI.
84
+
85
+
First, we describe the entire pipelne system's input and output. `.travis.yml` is the top level file.
86
+
87
+
* Input of the pipeline: `qemu-user-static-X.Y.Z-R.fcNN.$arch.rpm` RPM file under [Fedora Project URL](https://kojipkgs.fedoraproject.org/packages/qemu). Right now `$arch` is only x86_64.
88
+
* Output of the pipeline:
89
+
*[GitHub Releases page](https://github.com/multiarch/qemu-user-static/releases): `qemu-$arch-static` binary files, `qemu-$arch-static.tar.gz` and `x86_64_qemu-$arch-static.tar.gz` (`$from_arch_qemu-$arch-statc.tar.gz`). `qemu-$arch-static.tar.gz` files are same content with `x86_64_qemu-$arch-static.tar.gz`. It is an implementation to add supported host architectures `$from_arch` in the future.
90
+
* Images on [Docker Hub](https://hub.docker.com/r/multiarch/qemu-user-static/). For actual images, see `README.md` Usage - multiarch/qemu-user-static images section.
91
+
92
+
Second, we describe each program's input and output by sequence.
93
+
94
+
| Step | Name | Description | Input | Output |
95
+
| ---- | ---- | ----------- | ----- | ------ |
96
+
| 1 |`generate_tarballs.sh`| Create tar.gz files to upload GitHub Releases page. |`qemu-$arch-static` files in qemu-user-static RPM |`qemu-$arch-static.tar.gz` and `x86_64_qemu-$arch-static.tar.gz` files |
97
+
| 2 |`publish.sh`| Upload the tar.gz files by [GitHub API](https://developer.github.com/)|`qemu-$arch-static.tar.gz` and `x86_64_qemu-$arch-static.tar.gz` files |`qemu-$arch-static.tar.gz` and `x86_64_qemu-$arch-static.tar.gz` files on GitHub Releases page. |
98
+
| 3 |`update.sh`| Create container images on local |`x86_64_qemu-$arch-static.tar.gz` files on GitHub Releases page. | Container images on local |
99
+
| 4 |`test.sh`| Test created container images on local | Container images on local |`test/*` container images created as a result of tests |
100
+
| 5 |`docker push $DOCKER_REPO`| Push the container images to DockerHub |`multiarch/qemu-user-static:*` container images on local |`multiarch/qemu-user-static:*` container images on Docker Hub |
0 commit comments