Skip to content

[buildomat] Package all global zone tarballs into single tarball #2475

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Mar 7, 2023
Merged

[buildomat] Package all global zone tarballs into single tarball #2475

Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
77fb889
Package all globzl zone tarballs into single tarball
smklein Mar 3, 2023
eced651
switch variant asic
smklein Mar 3, 2023
170ee34
update name of global zone packages
smklein Mar 3, 2023
2b63d56
More attempts to correctly use this assembled GZ package
smklein Mar 3, 2023
748b0e9
try again with supplementary package
smklein Mar 3, 2023
3d34f05
/opt/oxide paths
smklein Mar 3, 2023
dcc9f6e
Pfexec
smklein Mar 4, 2023
6e83601
I forgot maghemite
smklein Mar 4, 2023
0a195c1
jk it's called mg-ddm
smklein Mar 4, 2023
4c7127a
... are config files load-bearing to create virtual hw?
smklein Mar 4, 2023
8532c7e
fix typo
smklein Mar 6, 2023
8874d34
layered fs approach
smklein Mar 6, 2023
c9fda84
Merge branch 'main' into publish-gz
smklein Mar 6, 2023
0055c16
Specify image type
smklein Mar 6, 2023
73b4cec
path to config files
smklein Mar 6, 2023
3316604
Make the gz package generation purely additive for now
smklein Mar 6, 2023
c70be86
Merge branch 'main' into publish-gz
smklein Mar 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 62 additions & 3 deletions .github/buildomat/jobs/package.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,14 @@
#: rust_toolchain = "1.66.1"
#: output_rules = [
#: "=/work/package.tar.gz",
#: "=/work/global-zone-packages.tar.gz",
#: "=/work/zones/*.tar.gz",
#: ]
#:
#: [[publish]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See: https://github.com/oxidecomputer/buildomat#configuration , under publish specifically

I think this'll be necessary for the helios repo to be able to "pick up these output artifacts" by a particular commit of Omicron

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm doing that work in oxidecomputer/helios#56

#: series = "image"
#: name = "global-zone-packages"
#: from_output = "/out/global-zone-packages.tar.gz"

set -o errexit
set -o pipefail
Expand All @@ -17,10 +22,15 @@ set -o xtrace
cargo --version
rustc --version

# Build
ptime -m ./tools/install_builder_prerequisites.sh -yp
ptime -m ./tools/create_self_signed_cert.sh -yp
ptime -m cargo run --locked --release --bin omicron-package -- -t 'image_type=standard switch_variant=asic' package
ptime -m cargo run --locked --release --bin omicron-package -- -t 'image_type=standard switch_variant=stub' package

ptime -m cargo run --locked --release --bin omicron-package -- package
tarball_src_dir="$(pwd)/out"

# Assemble some utilities into a tarball that can be used by deployment
# phases of buildomat.

files=(
out/*.tar
Expand All @@ -29,6 +39,55 @@ files=(
target/release/omicron-package
tools/create_virtual_hardware.sh
)

ptime -m tar cvzf /work/package.tar.gz "${files[@]}"

# Assemble global zone files in a temporary directory.
if ! tmp=$(mktemp -d); then
exit 1
fi
trap 'cd /; rm -rf "$tmp"' EXIT

# Header file, identifying this is intended to be layered in the global zone.
# Within the ramdisk, this means that all files under "root/foo" should appear
# in the global zone as "/foo".
echo '{"v":"1","t":"layer"}' > "$tmp/oxide.json"

# Extract the sled-agent tarball for re-packaging into the layered GZ archive.
pkg_dir="$tmp/root/opt/oxide/sled-agent"
mkdir -p "$pkg_dir"
cd "$pkg_dir"
tar -xvfz "$tarball_src_dir/omicron-sled-agent.tar"
# Ensure that the manifest for the sled agent exists in a location where it may
# be automatically initialized.
mkdir -p "$tmp/root/lib/svc/manifest/site/"
mv pkg/manifest.xml "$tmp/root/lib/svc/manifest/site/sled-agent.xml"
cd -

# Extract the mg-ddm tarball for re-packaging into the layered GZ archive.
pkg_dir="$tmp/root/opt/oxide/mg-ddm"
mkdir -p "$pkg_dir"
cd "$pkg_dir"
tar -xvfz "$tarball_src_dir/maghemite.tar"
cd -

mkdir -p /work
cd "$tmp" && tar cvfz /work/global-zone-packages.tar.gz oxide.json root
cd -

# Assemble Zone Images into their respective output locations.
mkdir -p /work/zones
mv out/*.tar.gz /work/zones/
zones=(
out/clickhouse.tar.gz
out/cockroachdb.tar.gz
out/crucible-pantry.tar.gz
out/crucible.tar.gz
out/external-dns.tar.gz
out/internal-dns.tar.gz
out/omicron-nexus.tar.gz
out/oximeter-collector.tar.gz
out/propolis-server.tar.gz
out/switch-asic.tar.gz
out/switch-stub.tar.gz
)
cp "${zones[@]}" /work/zones/