Skip to content

Commit cc8b857

Browse files
jimmykarilyjimmykarily
authored
Copy over the osbuilder-tools image assets and merge with auroraboot (#110)
* Copy over the osbuilder-tools image assets and merge with auroraboot So we can deprecate the osbuilder-tools image in favor of this one. Part of kairos-io/kairos#1633 Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me> * Print command output when things fail Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me> * Build versioned image with earthly Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me> * Fix ARG being in the global scope and not visible in the stage this making the `auroraboot --version` command fail https://docs.docker.com/build/building/variables/#scoping Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me> * Set the correct arg in pipeline Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me> * Add back qemu (needed by some test) Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me> * Remove duplicate block in dockerfile and keep luet repos up to date Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me> * Support images in the form of "dir:", in build-iso Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me> * Let elemental handle the schema and create missing tmp dir Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me> * Remove not necessary flag it seems that it only fails when the user building is not the same as the owner of the .git directory: golang/go#53532 I only saw it when running: ``` docker run --rm -it -v $PWD:/work --workdir /work golang go build . ``` With `docker build` it just works (maybe that command handles the users differently?). In any case, it doesn't affect CI or anything else so I'll remove it. Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me> --------- Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
1 parent 6e2c15c commit cc8b857

30 files changed

+1176
-73
lines changed

‎.github/workflows/image.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ jobs:
6161
with:
6262
builder: ${{ steps.buildx.outputs.name }}
6363
build-args: |
64-
BINARY_VERSION=${{ steps.prep.outputs.binary_version }}
64+
VERSION=${{ steps.prep.outputs.binary_version }}
6565
context: ./
6666
file: ./Dockerfile
6767
platforms: linux/amd64,linux/arm64

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
examples/airgap/build
22
examples/airgap/data
33
dist/
4+
build/

‎Dockerfile‎

Lines changed: 85 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,24 @@
1-
ARG VERSION=v0.400.3
21
ARG LUET_VERSION=0.35.5
2+
ARG LEAP_VERSION=15.5
33

44
FROM quay.io/luet/base:$LUET_VERSION AS luet
55

66
FROM golang AS builder
7-
ARG BINARY_VERSION=v0.0.0
7+
ARG VERSION=v0.0.0
88
WORKDIR /work
99
ADD go.mod .
1010
ADD go.sum .
1111
RUN go mod download
1212
ADD . .
13-
RUN CGO_ENABLED=0 go build -ldflags "-X main.version=${BINARY_VERSION}" -o auroraboot
14-
15-
FROM quay.io/kairos/osbuilder-tools:$VERSION
13+
ENV CGO_ENABLED=0
14+
ENV VERSION=$VERSION
15+
RUN go build -ldflags "-X main.version=${VERSION}" -o auroraboot
1616

17+
FROM opensuse/leap:$LEAP_VERSION AS default
18+
RUN zypper ref && zypper dup -y
19+
## ISO+ Arm image + Netboot + cloud images Build depedencies
20+
RUN zypper ref && zypper in -y bc qemu qemu-tools jq cdrtools docker git curl gptfdisk kpartx sudo xfsprogs parted binutils \
21+
util-linux-systemd e2fsprogs curl util-linux udev rsync grub2 dosfstools grub2-x86_64-efi squashfs mtools xorriso lvm2 zstd
1722
COPY --from=luet /usr/bin/luet /usr/bin/luet
1823
ENV LUET_NOLOCK=true
1924
ENV TMPDIR=/tmp
@@ -27,6 +32,80 @@ RUN cp /tmp/luet-${TARGETARCH}.yaml /etc/luet/luet.yaml
2732
## Uki artifacts, will be set under the /usr/kairos directory
2833
RUN luet install -y system/systemd-boot
2934

30-
RUN zypper in -y qemu binutils
35+
## Live CD artifacts
36+
RUN luet install -y livecd/grub2 --system-target /grub2
37+
RUN luet install -y livecd/grub2-efi-image --system-target /efi
38+
39+
## RPI64
40+
RUN luet install -y firmware/u-boot-rpi64 firmware/raspberrypi-firmware firmware/raspberrypi-firmware-config firmware/raspberrypi-firmware-dt --system-target /rpi/
41+
42+
## PineBook64 Pro
43+
RUN luet install -y arm-vendor-blob/u-boot-rockchip --system-target /pinebookpro/u-boot
44+
45+
## Odroid fw
46+
RUN luet install -y firmware/odroid-c2 --system-target /firmware/odroid-c2
47+
48+
## RAW images for current arch
49+
RUN luet install -y static/grub-efi --system-target /raw/grub
50+
RUN luet install -y static/grub-config --system-target /raw/grubconfig
51+
RUN luet install -y static/grub-artifacts --system-target /raw/grubartifacts
52+
53+
## RAW images for arm64
54+
# Luet will install this artifacts from the current arch repo, so in x86 it will
55+
# get them from the x86 repo and we want it to do it from the arm64 repo, even on x86
56+
# so we use the arm64 luet config and use that to install those on x86
57+
# This is being used by the prepare_arm_images.sh and build-arch-image.sh scripts
58+
RUN luet install --config /tmp/luet-arm64.yaml -y static/grub-efi --system-target /arm/raw/grubefi
59+
RUN luet install --config /tmp/luet-arm64.yaml -y static/grub-config --system-target /arm/raw/grubconfig
60+
RUN luet install --config /tmp/luet-arm64.yaml -y static/grub-artifacts --system-target /arm/raw/grubartifacts
61+
62+
# kairos-agent so we can use the pull-image
63+
RUN luet install -y system/kairos-agent
64+
65+
# remove luet tmp files. Side effect of setting the system-target is that it treats it as a root fs
66+
# so temporal files are stored in each dir
67+
RUN rm -Rf /grub2/var/tmp
68+
RUN rm -Rf /grub2/var/cache
69+
RUN rm -Rf /efi/var/tmp
70+
RUN rm -Rf /efi/var/cache
71+
RUN rm -Rf /rpi/var/tmp
72+
RUN rm -Rf /rpi/var/cache
73+
RUN rm -Rf /pinebookpro/u-boot/var/tmp
74+
RUN rm -Rf /pinebookpro/u-boot/var/cache
75+
RUN rm -Rf /firmware/odroid-c2/var/tmp
76+
RUN rm -Rf /firmware/odroid-c2/var/cache
77+
RUN rm -Rf /raw/grub/var/tmp
78+
RUN rm -Rf /raw/grub/var/cache
79+
RUN rm -Rf /raw/grubconfig/var/tmp
80+
RUN rm -Rf /raw/grubconfig/var/cache
81+
RUN rm -Rf /raw/grubartifacts/var/tmp
82+
RUN rm -Rf /raw/grubartifacts/var/cache
83+
RUN rm -Rf /arm/raw/grubefi/var/tmp
84+
RUN rm -Rf /arm/raw/grubefi/var/cache
85+
RUN rm -Rf /arm/raw/grubconfig/var/tmp
86+
RUN rm -Rf /arm/raw/grubconfig/var/cache
87+
RUN rm -Rf /arm/raw/grubartifacts/var/tmp
88+
RUN rm -Rf /arm/raw/grubartifacts/var/cache
89+
90+
# ISO build config
91+
COPY ./image-assets/add-cloud-init.sh /add-cloud-init.sh
92+
COPY ./image-assets/kairos-release.tmpl /kairos-release.tmpl
93+
COPY ./image-assets/ipxe.tmpl /ipxe.tmpl
94+
COPY ./image-assets/update-os-release.sh /update-os-release.sh
95+
96+
# ARM helpers
97+
COPY ./image-assets/build-arm-image.sh /build-arm-image.sh
98+
COPY ./image-assets/arm /arm
99+
COPY ./image-assets/prepare_arm_images.sh /prepare_arm_images.sh
100+
101+
# RAW images helpers
102+
COPY ./image-assets/gce.sh /gce.sh
103+
COPY ./image-assets/raw-images.sh /raw-images.sh
104+
COPY ./image-assets/azure.sh /azure.sh
105+
COPY ./image-assets/netboot.sh /netboot.sh
106+
107+
COPY ./image-assets/defaults.yaml /defaults.yaml
108+
31109
COPY --from=builder /work/auroraboot /usr/bin/auroraboot
110+
32111
ENTRYPOINT ["/usr/bin/auroraboot"]

‎Earthfile‎

Lines changed: 38 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,25 @@
11
VERSION 0.7
2-
ARG --global OSBUILDER_VERSION=v0.9.0
32
ARG --global GO_VERSION=1.23-bookworm
43

54
# renovate: datasource=github-releases depName=kairos-io/kairos
65
ARG IMAGE_VERSION=v3.2.1
76
ARG --global BASE_IMAGE=quay.io/kairos/ubuntu:24.04-core-amd64-generic-${IMAGE_VERSION}-uki
87

8+
version:
9+
FROM alpine
10+
RUN apk update && apk add git
11+
12+
COPY . .
13+
RUN --no-cache git describe --always --tags --dirty > VERSION
14+
SAVE ARTIFACT VERSION VERSION
15+
916
image:
10-
FROM DOCKERFILE --build-arg VERSION=$OSBUILDER_VERSION -f Dockerfile .
11-
RUN zypper in -y qemu
17+
FROM +version
18+
ARG VERSION=$(cat VERSION)
19+
20+
FROM DOCKERFILE --build-arg VERSION=$VERSION -f Dockerfile .
21+
22+
SAVE IMAGE quay.io/kairos/auroraboot:$VERSION
1223

1324
test-label:
1425
FROM alpine
@@ -71,3 +82,27 @@ test-bootable:
7182
ARG CREATE_VM=true
7283
RUN date
7384
RUN go run github.com/onsi/ginkgo/v2/ginkgo run --label-filter "bootable" -v --fail-fast -r ./e2e
85+
86+
last-commit-packages:
87+
FROM quay.io/skopeo/stable
88+
RUN dnf install -y jq
89+
WORKDIR build
90+
ENV jqQuery='.Tags | map(select(. | contains("-repository.yaml"))) | sort_by(. | sub("v";"") | sub("-repository.yaml";"") | sub("-git.*";"") | .[0:12] | tonumber) | .[-1]'
91+
RUN skopeo list-tags docker://quay.io/kairos/packages | jq -rc "${jqQuery}" > REPO_AMD64
92+
RUN skopeo list-tags docker://quay.io/kairos/packages-arm64 | jq -rc "${jqQuery}" > REPO_ARM64
93+
SAVE ARTIFACT REPO_AMD64 REPO_AMD64
94+
SAVE ARTIFACT REPO_ARM64 REPO_ARM64
95+
96+
bump-repositories:
97+
FROM mikefarah/yq
98+
WORKDIR build
99+
COPY +last-commit-packages/REPO_AMD64 REPO_AMD64
100+
COPY +last-commit-packages/REPO_ARM64 REPO_ARM64
101+
ARG REPO_AMD64=$(cat REPO_AMD64)
102+
ARG REPO_ARM64=$(cat REPO_ARM64)
103+
COPY image-assets/luet-amd64.yaml luet-amd64.yaml
104+
COPY image-assets/luet-arm64.yaml luet-arm64.yaml
105+
RUN yq eval ".repositories[0] |= . * { \"reference\": \"${REPO_AMD64}\" }" -i luet-amd64.yaml
106+
RUN yq eval ".repositories[0] |= . * { \"reference\": \"${REPO_ARM64}\" }" -i luet-arm64.yaml
107+
SAVE ARTIFACT luet-arm64.yaml AS LOCAL image-assets/luet-arm64.yaml
108+
SAVE ARTIFACT luet-amd64.yaml AS LOCAL image-assets/luet-amd64.yaml

‎deployer/register.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ const (
1919
opPrepareNetboot = "prepare-netboot"
2020
opStartNetboot = "start-netboot"
2121

22-
opContainerPull = "container-pull"
22+
opDumpSource = "dump-source"
2323
opGenISO = "gen-iso"
2424
opPreparetmproot = "prepare-temp"
2525
opExtractNetboot = "extract-netboot"
@@ -43,7 +43,7 @@ func RegisterAll(d *Deployer) error {
4343
d.StepPrepNetbootDir,
4444
d.StepPrepISODir,
4545
d.StepCopyCloudConfig,
46-
d.StepPullContainer,
46+
d.StepDumpSource,
4747
d.StepGenISO,
4848
d.StepExtractNetboot,
4949
//TODO: add Validate step

‎deployer/steps.go‎

Lines changed: 6 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,6 @@ import (
44
"context"
55
"os"
66
"path/filepath"
7-
"strings"
87

98
"github.com/kairos-io/AuroraBoot/pkg/ops"
109
"github.com/spectrocloud-labs/herd"
@@ -21,7 +20,7 @@ func (d *Deployer) StepPrepNetbootDir() error {
2120
func (d *Deployer) StepPrepTmpRootDir() error {
2221
return d.Add(opPreparetmproot, herd.WithCallback(
2322
func(ctx context.Context) error {
24-
return os.MkdirAll(d.dstNetboot(), 0700)
23+
return os.MkdirAll(d.tmpRootFs(), 0700)
2524
},
2625
))
2726
}
@@ -40,17 +39,17 @@ func (d *Deployer) StepCopyCloudConfig() error {
4039
}))
4140
}
4241

43-
func (d *Deployer) StepPullContainer() error {
42+
func (d *Deployer) StepDumpSource() error {
4443
// Ops to generate from container image
45-
return d.Add(opContainerPull,
44+
return d.Add(opDumpSource,
4645
herd.EnableIf(d.fromImage),
47-
herd.WithDeps(opPreparetmproot), herd.WithCallback(ops.PullContainerImage(d.containerImage(), d.tmpRootFs())))
46+
herd.WithDeps(opPreparetmproot), herd.WithCallback(ops.DumpSource(d.Artifact.ContainerImage, d.tmpRootFs())))
4847
}
4948

5049
func (d *Deployer) StepGenISO() error {
5150
return d.Add(opGenISO,
5251
herd.EnableIf(func() bool { return d.fromImage() && !d.rawDiskIsSet() && d.Config.Disk.ARM == nil }),
53-
herd.WithDeps(opContainerPull, opCopyCloudConfig), herd.WithCallback(ops.GenISO(d.tmpRootFs(), d.destination(), d.Config.ISO)))
52+
herd.WithDeps(opDumpSource, opCopyCloudConfig), herd.WithCallback(ops.GenISO(d.tmpRootFs(), d.destination(), d.Config.ISO)))
5453
}
5554

5655
func (d *Deployer) StepExtractNetboot() error {
@@ -178,16 +177,6 @@ func (d *Deployer) StepStartNetboot() error {
178177
)
179178
}
180179

181-
func (d *Deployer) containerImage() string {
182-
// Pull local docker daemon if container image starts with docker://
183-
containerImage := d.Artifact.ContainerImage
184-
if strings.HasPrefix(containerImage, "docker://") {
185-
containerImage = strings.ReplaceAll(containerImage, "docker://", "")
186-
}
187-
188-
return containerImage
189-
}
190-
191180
func (d *Deployer) fromImage() bool {
192181
return d.Artifact.ContainerImage != ""
193182
}
@@ -233,7 +222,7 @@ func (d *Deployer) isoOption() bool {
233222
}
234223

235224
func (d *Deployer) imageOrSquashFS() herd.OpOption {
236-
return herd.IfElse(d.fromImage(), herd.WithDeps(opContainerPull), herd.WithDeps(opExtractSquashFS))
225+
return herd.IfElse(d.fromImage(), herd.WithDeps(opDumpSource), herd.WithDeps(opExtractSquashFS))
237226
}
238227

239228
func (d *Deployer) cloudConfigPath() string {

‎go.mod‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@ godebug x509negativeserial=1
99
require (
1010
github.com/cavaliergopher/grab/v3 v3.0.1
1111
github.com/containerd/containerd v1.7.23
12-
github.com/distribution/reference v0.6.0
1312
github.com/foxboron/go-uefi v0.0.0-20241017190036-fab4fdf2f2f3
1413
github.com/foxboron/sbctl v0.0.0-20240526163235-64e649b31c8e
1514
github.com/gofrs/uuid v4.4.0+incompatible
@@ -71,6 +70,7 @@ require (
7170
github.com/cyphar/filepath-securejoin v0.2.4 // indirect
7271
github.com/denisbrodbeck/machineid v1.0.1 // indirect
7372
github.com/diskfs/go-diskfs v1.4.2 // indirect
73+
github.com/distribution/reference v0.6.0 // indirect
7474
github.com/djherbis/times v1.6.0 // indirect
7575
github.com/docker/cli v27.1.1+incompatible // indirect
7676
github.com/docker/distribution v2.8.2+incompatible // indirect

‎image-assets/add-cloud-init.sh‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#!/bin/bash
2+
# docker run --entrypoint /add-cloud-init.sh -v $PWD:/work -ti --rm test https://github.com/kairos-io/kairos/releases/download/v1.1.2/kairos-alpine-v1.1.2.iso /work/test.iso /work/config.yaml
3+
4+
set -ex
5+
6+
ISO=$1
7+
OUT=$2
8+
CONFIG=$3
9+
10+
case ${ISO} in
11+
http*)
12+
curl -L "${ISO}" -o in.iso
13+
ISO=in.iso
14+
;;
15+
esac
16+
17+
# Needs xorriso >=1.5.4
18+
xorriso -indev $ISO -outdev $OUT -map $CONFIG /config.yaml -boot_image any replay

‎image-assets/arm/boards/odroid_c2.sh‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#!/bin/bash
2+
3+
image=$1
4+
5+
if [ -z "$image" ]; then
6+
echo "No image specified"
7+
exit 1
8+
fi
9+
10+
# conv=notrunc ?
11+
dd if=/firmware/odroid-c2/bl1.bin.hardkernel of=$image conv=fsync bs=1 count=442
12+
dd if=/firmware/odroid-c2/bl1.bin.hardkernel of=$image conv=fsync bs=512 skip=1 seek=1
13+
dd if=/firmware/odroid-c2/u-boot.odroidc2 of=$image conv=fsync bs=512 seek=97

‎image-assets/arm/boards/pinebookpro.sh‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#!/bin/bash
2+
3+
image=$1
4+
5+
if [ -z "$image" ]; then
6+
echo "No image specified"
7+
exit 1
8+
fi
9+
10+
LOADER_OFFSET=${LOADER_OFFSET:-"64"}
11+
LOADER_IMAGE=${LOADER_IMAGE:-"idbloader.img"}
12+
UBOOT_IMAGE=${UBOOT_IMAGE:-"u-boot.itb"}
13+
UBOOT_OFFSET=${UBOOT_OFFSET:-"16384"}
14+
15+
echo "Writing idbloader"
16+
dd conv=notrunc if=/pinebookpro/u-boot/usr/lib/u-boot/pinebook-pro-rk3399/${LOADER_IMAGE} of="$image" conv=fsync seek=${LOADER_OFFSET}
17+
echo "Writing u-boot image"
18+
dd conv=notrunc if=/pinebookpro/u-boot/usr/lib/u-boot/pinebook-pro-rk3399/${UBOOT_IMAGE} of="$image" conv=fsync seek=${UBOOT_OFFSET}
19+
sync $image

0 commit comments

Comments
 (0)