Skip to content

Commit caa9518

Browse files
jacobweinstockjacobweinstock
authored
Updates: (#217)
## Description <!--- Please describe what this PR is going to change --> - Use latest v1.0.0 linuxkit images. - Add images dir with runc and containerd to be able to better control the version. - Use latest upstream init as it has cgroups v2 as the default. - Add dhcp to the linuxkit services so that interfaces stay configured after lease times expire. - Update to latest Docker in Docker version and update linuxkit spec to handle cgroups v2. - Increase the kernel command line parameter size to 4096 bytes to allow for longer things to be passed like certs, etc. - Default to linuxkit 1.2.0 binary as the containerd issues are resolved with the ccgroup v2 changes. ## Why is this needed <!--- Link to issue you have raised --> Fixes: # ## How Has This Been Tested? <!--- Please describe in detail how you tested your changes. --> <!--- Include details of your testing environment, and the tests you ran to --> <!--- see how your change affects other areas of the code, etc. --> ## How are existing users impacted? What migration steps/scripts do we need? <!--- Fixes a bug, unblocks installation, removes a component of the stack etc --> <!--- Requires a DB migration script, etc. --> ## Checklist: I have: - [ ] updated the documentation and/or roadmap (if required) - [ ] added unit or e2e tests - [ ] provided instructions on how to upgrade
2 parents 2679d77 + f15ec0c commit caa9518

File tree

22 files changed

+258
-125
lines changed

22 files changed

+258
-125
lines changed

‎.github/mergify.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ queue_rules:
33
conditions:
44
# Conditions to get out of the queue (= merged)
55
- check-success=DCO
6+
- check-success~="^Hook .*"
7+
- check-success~="^Kernel .*"
68

79
pull_request_rules:
810
- name: Automatic merge on approval
@@ -12,7 +14,8 @@ pull_request_rules:
1214
- "#changes-requested-reviews-by=0"
1315
- "#review-requested=0"
1416
- check-success=DCO
15-
- check-success~=validation$
17+
- check-success~="^Hook .*"
18+
- check-success~="^Kernel .*"
1619
- label!=do-not-merge
1720
- label=ready-to-merge
1821
actions:

‎.github/workflows/build-all-matrix.yaml‎

Lines changed: 16 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ on:
44
# # every day at 5am UTC
55
# - cron: '0 5 * * *'
66
workflow_dispatch:
7+
pull_request:
8+
push:
79

810
env: # Global environment, passed to all jobs & all steps
911
# Default to quay.io, which is also the default for the CLI.
@@ -24,6 +26,12 @@ env: # Global environment, passed to all jobs & all steps
2426

2527
# GHA runner configuration. See bash/json-matrix.sh for more details.
2628
CI_RUNNER_LK_CONTAINERS_ARM64: "ARM64" # Use a self-hosted runner with the "ARM64" tag for the ARM64 builds of LK containers
29+
CI_RUNNER_LK_CONTAINERS_AMD64: "X64" # Use a self-hosted runner with the "X86" tag for the AMD64 builds of LK containers
30+
CI_RUNNER_LK_ARM64: "ARM64" # Use a self-hosted runner with the "ARM64" tag for the ARM64 linuxkit builds
31+
CI_RUNNER_LK_AMD64: "X64" # Use a self-hosted runner with the "X86" tag for the AMD64 linuxkit builds
32+
CI_RUNNER_KERNEL_AMD64: "X64" # Use a self-hosted runner with the "X86" tag for the AMD64 kernel builds
33+
CI_RUNNER_KERNEL_ARM64: "ARM64" # Use a self-hosted runner with the "ARM64" tag for the ARM64 kernel builds
34+
2735

2836
jobs:
2937

@@ -66,7 +74,7 @@ jobs:
6674
uses: docker/setup-buildx-action@v3
6775

6876
- name: Docker Login to quay.io
69-
if: ${{ env.REGISTRY == 'quay.io' }}
77+
if: ${{ env.REGISTRY == 'quay.io' && github.ref == 'refs/heads/main' }}
7078
uses: docker/login-action@v3
7179
with: { registry: "quay.io", username: "${{ secrets.QUAY_USERNAME }}", password: "${{ secrets.QUAY_PASSWORD }}" }
7280

@@ -78,7 +86,7 @@ jobs:
7886
- name: Build and Push LinuxKit containers for ${{matrix.docker_arch}}
7987
env:
8088
DOCKER_ARCH: "${{ matrix.docker_arch }}"
81-
DO_PUSH: "yes"
89+
DO_PUSH: "${{ github.ref == 'refs/heads/main' && 'yes' || 'no' }}"
8290
run: bash build.sh linuxkit-containers
8391

8492
build-kernels:
@@ -97,7 +105,7 @@ jobs:
97105
uses: docker/setup-buildx-action@v3
98106

99107
- name: Docker Login to quay.io
100-
if: ${{ env.REGISTRY == 'quay.io' }}
108+
if: ${{ env.REGISTRY == 'quay.io' && github.ref == 'refs/heads/main' }}
101109
uses: docker/login-action@v3
102110
with: { registry: "quay.io", username: "${{ secrets.QUAY_USERNAME }}", password: "${{ secrets.QUAY_PASSWORD }}" }
103111

@@ -108,7 +116,7 @@ jobs:
108116

109117
- name: Build and push Kernel ${{matrix.kernel}} (${{ matrix.arch }})
110118
env:
111-
DO_PUSH: "yes"
119+
DO_PUSH: "${{ github.ref == 'refs/heads/main' && 'yes' || 'no' }}"
112120
run: bash build.sh build-kernel "${{ matrix.kernel }}"
113121

114122
build-hook-ensemble:
@@ -127,12 +135,12 @@ jobs:
127135
uses: docker/setup-buildx-action@v3
128136

129137
- name: Docker Login to DockerHub # read-only token, required to be able to pull all the linuxkit pkgs without getting rate limited.
130-
if: ${{ env.LOGIN_TO_DOCKERHUB == 'yes' }}
138+
if: ${{ env.LOGIN_TO_DOCKERHUB == 'yes' && github.ref == 'refs/heads/main' }}
131139
uses: docker/login-action@v3
132140
with: { registry: "docker.io", username: "${{ secrets.DOCKERHUB_USERNAME }}", password: "${{ secrets.DOCKERHUB_PASSWORD }}" }
133141

134142
- name: Docker Login to quay.io
135-
if: ${{ env.REGISTRY == 'quay.io' }}
143+
if: ${{ env.REGISTRY == 'quay.io' && github.ref == 'refs/heads/main' }}
136144
uses: docker/login-action@v3
137145
with: { registry: "quay.io", username: "${{ secrets.QUAY_USERNAME }}", password: "${{ secrets.QUAY_PASSWORD }}" }
138146

@@ -178,7 +186,6 @@ jobs:
178186
with:
179187
pattern: "hook-tarball-*"
180188
merge-multiple: true
181-
path: out
182189

183190
- name: Figure Out Commit Short ID
184191
id: commitid
@@ -219,15 +226,15 @@ jobs:
219226
uses: jmgilman/actions-generate-checksum@v1
220227
with:
221228
method: sha512
222-
patterns: out/*.tar.gz
229+
patterns: "*.tar.gz"
223230

224231
- name: Update latest release
225232
uses: softprops/action-gh-release@v2
226233
with:
227234
name: Hook Latest Development Build
228235
body: ${{env.RELEASE_NOTES}}
229236
files: |
230-
out/*.tar.gz
237+
*.tar.gz
231238
checksum.txt
232239
prerelease: true
233240
tag_name: latest

‎bash/hook-lk-containers.sh‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,13 @@ function build_all_hook_linuxkit_containers() {
44
log info "Building all LinuxKit containers..."
55
: "${DOCKER_ARCH:?"ERROR: DOCKER_ARCH is not defined"}"
66

7-
build_hook_linuxkit_container hook-bootkit HOOK_CONTAINER_BOOTKIT_IMAGE
8-
build_hook_linuxkit_container hook-docker HOOK_CONTAINER_DOCKER_IMAGE
9-
build_hook_linuxkit_container hook-mdev HOOK_CONTAINER_MDEV_IMAGE
7+
# when adding new container builds here you'll also want to add them to the
8+
# `linuxkit_build` function in the linuxkit.sh file.
9+
build_hook_linuxkit_container images/hook-bootkit HOOK_CONTAINER_BOOTKIT_IMAGE
10+
build_hook_linuxkit_container images/hook-docker HOOK_CONTAINER_DOCKER_IMAGE
11+
build_hook_linuxkit_container images/hook-mdev HOOK_CONTAINER_MDEV_IMAGE
12+
build_hook_linuxkit_container images/containerd HOOK_CONTAINER_CONTAINERD_IMAGE
13+
build_hook_linuxkit_container images/runc HOOK_CONTAINER_RUNC_IMAGE
1014
}
1115

1216
function build_hook_linuxkit_container() {

‎bash/json-matrix.sh‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -47,12 +47,12 @@ function output_gha_matrixes() {
4747
prepare_json_matrix_lkcontainers "LK_CONTAINERS" # reads all_arches's keys and sets full_json
4848
declare lkcontainers_json="${full_json}"
4949

50-
# If under GHA, set a GHA output variable, otherwise just log them
51-
if [[ -z "${GITHUB_OUTPUT}" ]]; then
52-
log info "Would have set GHA output kernels_json to: ${kernels_json}"
53-
log info "Would have set GHA output lk_hooks_json to: ${lk_hooks_json}"
54-
log info "Would have set GHA output lkcontainers_json to: ${lkcontainers_json}"
55-
else
50+
log info "kernels_json to: ${kernels_json}"
51+
log info "lk_hooks_json to: ${lk_hooks_json}"
52+
log info "lkcontainers_json to: ${lkcontainers_json}"
53+
54+
# If under GHA, set a GHA output variable.
55+
if [[ -n "${GITHUB_OUTPUT}" ]]; then
5656
# shellcheck disable=SC2129 # no, thanks, shellcheck.
5757
echo "kernels_json=${kernels_json}" >> "${GITHUB_OUTPUT}"
5858
echo "lk_hooks_json=${lk_hooks_json}" >> "${GITHUB_OUTPUT}"

‎bash/linuxkit.sh‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ function linuxkit_build() {
5050
fi
5151

5252
# Build the containers in this repo used in the LinuxKit YAML;
53-
build_all_hook_linuxkit_containers # sets HOOK_CONTAINER_BOOTKIT_IMAGE, HOOK_CONTAINER_DOCKER_IMAGE, HOOK_CONTAINER_MDEV_IMAGE
53+
build_all_hook_linuxkit_containers # sets HOOK_CONTAINER_BOOTKIT_IMAGE, HOOK_CONTAINER_DOCKER_IMAGE, HOOK_CONTAINER_MDEV_IMAGE, HOOK_CONTAINER_CONTAINERD_IMAGE
5454

5555
# Template the linuxkit configuration file.
5656
# - You'd think linuxkit would take --build-args or something by now, but no.
@@ -67,7 +67,9 @@ function linuxkit_build() {
6767
HOOK_CONTAINER_BOOTKIT_IMAGE="${HOOK_CONTAINER_BOOTKIT_IMAGE}" \
6868
HOOK_CONTAINER_DOCKER_IMAGE="${HOOK_CONTAINER_DOCKER_IMAGE}" \
6969
HOOK_CONTAINER_MDEV_IMAGE="${HOOK_CONTAINER_MDEV_IMAGE}" \
70-
envsubst '$HOOK_VERSION $HOOK_KERNEL_IMAGE $HOOK_KERNEL_ID $HOOK_KERNEL_VERSION $HOOK_CONTAINER_BOOTKIT_IMAGE $HOOK_CONTAINER_DOCKER_IMAGE $HOOK_CONTAINER_MDEV_IMAGE' \
70+
HOOK_CONTAINER_CONTAINERD_IMAGE="${HOOK_CONTAINER_CONTAINERD_IMAGE}" \
71+
HOOK_CONTAINER_RUNC_IMAGE="${HOOK_CONTAINER_RUNC_IMAGE}" \
72+
envsubst '$HOOK_VERSION $HOOK_KERNEL_IMAGE $HOOK_KERNEL_ID $HOOK_KERNEL_VERSION $HOOK_CONTAINER_BOOTKIT_IMAGE $HOOK_CONTAINER_DOCKER_IMAGE $HOOK_CONTAINER_MDEV_IMAGE $HOOK_CONTAINER_CONTAINERD_IMAGE $HOOK_CONTAINER_RUNC_IMAGE' \
7173
> "hook.${inventory_id}.yaml"
7274

7375
declare -g linuxkit_bin=""
@@ -139,7 +141,7 @@ function linuxkit_build() {
139141

140142
# tar the files into out/hook.tar in such a way that vmlinuz and initramfs are at the root of the tar; pigz it
141143
# Those are the artifacts published to the GitHub release
142-
tar -cvf- -C "out/hook" "${output_files[@]}" | pigz > "out/hook-${OUTPUT_ID}.tar.gz"
144+
tar -cvf- -C "out/hook" "${output_files[@]}" | pigz > "out/hook_${OUTPUT_ID}.tar.gz"
143145
}
144146

145147
function linuxkit_run_qemu() {

‎build.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ declare -g HOOK_LK_CONTAINERS_OCI_BASE="${HOOK_LK_CONTAINERS_OCI_BASE:-"quay.io/
2929
declare -g SKOPEO_IMAGE="${SKOPEO_IMAGE:-"quay.io/skopeo/stable:latest"}"
3030

3131
# See https://github.com/linuxkit/linuxkit/releases
32-
declare -g -r LINUXKIT_VERSION_DEFAULT="1.0.1" # LinuxKit version to use by default; each flavor can set its own too
32+
declare -g -r LINUXKIT_VERSION_DEFAULT="1.2.0" # LinuxKit version to use by default; each flavor can set its own too
3333

3434
# Directory to use for storing downloaded artifacts: LinuxKit binary, shellcheck binary, etc.
3535
declare -g -r CACHE_DIR="${CACHE_DIR:-"cache"}"

‎images/containerd/Dockerfile‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
FROM linuxkit/alpine:146f540f25cd92ec8ff0c5b0c98342a9a95e479e as builder
2+
3+
4+
# checkout and compile containerd
5+
# Update `FROM` in `pkg/containerd/Dockerfile`, `pkg/init/Dockerfile` and
6+
# `test/pkg/containerd/Dockerfile` when changing this.
7+
ENV CONTAINERD_REPO=https://github.com/containerd/containerd.git
8+
9+
ENV CONTAINERD_COMMIT=v1.7.15
10+
ENV GOPATH=/go
11+
RUN apk add go git
12+
RUN mkdir -p $GOPATH/src/github.com/containerd && \
13+
cd $GOPATH/src/github.com/containerd && \
14+
git clone https://github.com/containerd/containerd.git && \
15+
cd $GOPATH/src/github.com/containerd/containerd && \
16+
git checkout $CONTAINERD_COMMIT
17+
RUN apk add --no-cache btrfs-progs-dev gcc libc-dev linux-headers make libseccomp-dev
18+
WORKDIR $GOPATH/src/github.com/containerd/containerd
19+
RUN make binaries EXTRA_FLAGS="-buildmode pie" EXTRA_LDFLAGS='-extldflags "-fno-PIC -static"' BUILDTAGS="static_build no_devmapper"
20+
21+
RUN cp bin/containerd bin/ctr bin/containerd-shim bin/containerd-shim-runc-v2 /usr/bin/
22+
RUN strip /usr/bin/containerd /usr/bin/ctr /usr/bin/containerd-shim /usr/bin/containerd-shim-runc-v2
23+
24+
FROM scratch as containerd-dev
25+
ENTRYPOINT []
26+
WORKDIR /
27+
COPY --from=builder /usr/bin/containerd /usr/bin/ctr /usr/bin/containerd-shim /usr/bin/containerd-shim-runc-v2 /usr/bin/
28+
COPY --from=builder /go/src/github.com/containerd/containerd /go/src/github.com/containerd/containerd
29+
30+
# Dockerfile to build linuxkit/containerd for linuxkit
31+
FROM linuxkit/alpine:146f540f25cd92ec8ff0c5b0c98342a9a95e479e as alpine
32+
33+
RUN apk add tzdata binutils
34+
RUN mkdir -p /etc/init.d && ln -s /usr/bin/service /etc/init.d/020-containerd
35+
36+
FROM containerd-dev
37+
38+
FROM scratch
39+
ENTRYPOINT []
40+
WORKDIR /
41+
COPY --from=containerd-dev /usr/bin/containerd /usr/bin/ctr /usr/bin/containerd-shim /usr/bin/containerd-shim-runc-v2 /usr/bin/
42+
COPY --from=alpine /usr/share/zoneinfo/UTC /etc/localtime
43+
COPY --from=alpine /etc/init.d/ /etc/init.d/
44+
COPY etc etc/
45+
COPY --from=alpine /etc/apk /etc/apk/
46+
COPY --from=alpine /lib/apk /lib/apk/

‎images/containerd/etc/containerd/config.toml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
state = "/run/containerd"
2+
root = "/var/lib/containerd"
3+
disabled_plugins = ["cri"]
4+
5+
[grpc]
6+
address = "/run/containerd/containerd.sock"
7+
uid = 0
8+
gid = 0
9+
10+
[debug]
11+
address = "/run/containerd/debug.sock"
12+
level = "info"
13+
14+
[metrics]
15+
address = ""

‎hook-bootkit/Dockerfile‎ renamed to ‎images/hook-bootkit/Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM golang:1.20-alpine as dev
1+
FROM golang:1.21-alpine as dev
22
COPY . /src/
33
WORKDIR /src
44
RUN go mod download

‎hook-bootkit/go.mod‎ renamed to ‎images/hook-bootkit/go.mod‎

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,30 +3,39 @@ module github.com/tinkerbell/hook/hook-bootkit
33
go 1.17
44

55
require (
6-
github.com/cenkalti/backoff/v4 v4.2.1
7-
github.com/docker/docker v24.0.4+incompatible
8-
github.com/go-logr/logr v1.2.4
6+
github.com/cenkalti/backoff/v4 v4.3.0
7+
github.com/docker/docker v26.1.0+incompatible
8+
github.com/go-logr/logr v1.4.1
99
github.com/go-logr/zerologr v1.2.3
1010
github.com/rs/zerolog v1.30.0
1111
)
1212

1313
require (
1414
github.com/Microsoft/go-winio v0.6.1 // indirect
15-
github.com/docker/distribution v2.8.2+incompatible // indirect
15+
github.com/containerd/log v0.1.0 // indirect
16+
github.com/distribution/reference v0.6.0 // indirect
1617
github.com/docker/go-connections v0.4.0 // indirect
1718
github.com/docker/go-units v0.5.0 // indirect
19+
github.com/felixge/httpsnoop v1.0.4 // indirect
20+
github.com/go-logr/stdr v1.2.2 // indirect
1821
github.com/gogo/protobuf v1.3.2 // indirect
1922
github.com/mattn/go-colorable v0.1.12 // indirect
2023
github.com/mattn/go-isatty v0.0.14 // indirect
24+
github.com/moby/docker-image-spec v1.3.1 // indirect
2125
github.com/moby/term v0.5.0 // indirect
2226
github.com/morikuni/aec v1.0.0 // indirect
2327
github.com/opencontainers/go-digest v1.0.0 // indirect
2428
github.com/opencontainers/image-spec v1.0.2 // indirect
2529
github.com/pkg/errors v0.9.1 // indirect
26-
github.com/stretchr/testify v1.8.4 // indirect
30+
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.51.0 // indirect
31+
go.opentelemetry.io/otel v1.26.0 // indirect
32+
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.26.0 // indirect
33+
go.opentelemetry.io/otel/metric v1.26.0 // indirect
34+
go.opentelemetry.io/otel/sdk v1.26.0 // indirect
35+
go.opentelemetry.io/otel/trace v1.26.0 // indirect
2736
golang.org/x/mod v0.8.0 // indirect
28-
golang.org/x/net v0.12.0 // indirect
29-
golang.org/x/sys v0.10.0 // indirect
37+
golang.org/x/net v0.23.0 // indirect
38+
golang.org/x/sys v0.19.0 // indirect
3039
golang.org/x/time v0.3.0 // indirect
3140
golang.org/x/tools v0.6.0 // indirect
3241
gotest.tools/v3 v3.5.0 // indirect

0 commit comments

Comments
 (0)