-
Notifications
You must be signed in to change notification settings - Fork 531
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updated Builder script #234
Open
moos3
wants to merge
1
commit into
gliderlabs:master
Choose a base branch
from
moos3:update-builder
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
95fea88f184acb0a557a0c9538c93b510b87f83a | ||
dc0ba70a45f554c38e5cd237c31f4d983eed9586 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,5 @@ | ||
FROM alpine:3.2 | ||
FROM alpine:base | ||
COPY scripts/mkimage-alpine.bash scripts/apk-install / | ||
RUN apk update | ||
RUN /apk-install bash tzdata | ||
ENTRYPOINT ["/mkimage-alpine.bash"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
FROM scratch | ||
ADD rootfs.tar.gz / |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,219 @@ | ||
#!/bin/bash | ||
set -e | ||
|
||
####### | ||
## | ||
## This script will bootstrap a clean unadulterated alpine builder. | ||
## Author: moos3 <richard@guthnur.net> | ||
## | ||
####### | ||
|
||
### Update this keys as needed. | ||
key_sha256sums="9c102bcc376af1498d549b77bdbfa815ae86faa1d2d82f040e616b18ef2df2d4 alpine-devel@lists.alpinelinux.org-4a6a0840.rsa.pub | ||
2adcf7ce224f476330b5360ca5edb92fd0bf91c92d83292ed028d7c4e26333ab alpine-devel@lists.alpinelinux.org-4d07755e.rsa.pub | ||
ebf31683b56410ecc4c00acd9f6e2839e237a3b62b5ae7ef686705c7ba0396a9 alpine-devel@lists.alpinelinux.org-5243ef4b.rsa.pub | ||
1bb2a846c0ea4ca9d0e7862f970863857fc33c32f5506098c636a62a726a847b alpine-devel@lists.alpinelinux.org-524d27bb.rsa.pub | ||
12f899e55a7691225603d6fb3324940fc51cd7f133e7ead788663c2b7eecb00c alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub" | ||
|
||
get_static_apk () { | ||
wget="wget -q -O -" | ||
pkglist=apk-tools-static:alpine-baselayout:musl-utils:apk-keys | ||
auto_repo_dir= | ||
|
||
if [ -z "$repository" ]; then | ||
url=http://dl-4.alpinelinux.org/alpine/ | ||
yaml_path="latest-stable/releases/$apk_arch/latest-releases.yaml" | ||
if [ -z "$release" ]; then | ||
echo -n "Determining the latest release... " | ||
release=$($wget $url/$yaml_path | \ | ||
awk '$1 == "branch:" {print $2; exit 0}') | ||
if [ -z "$release" ]; then | ||
release=$($wget $url/.latest.$apk_arch.txt | \ | ||
cut -d " " -f 3 | cut -d / -f 1 | uniq) | ||
fi | ||
if [ -z "$release" ]; then | ||
echo failed | ||
return 1 | ||
fi | ||
echo "Alpine release: $release" | ||
fi | ||
auto_repo_dir=$release/main | ||
repository=$url/$auto_repo_dir | ||
pkglist=$pkglist:alpine-mirrors | ||
fi | ||
|
||
echo "Using static apk from $repository/$apk_arch" | ||
wget="$wget $repository/$apk_arch" | ||
echo "running $wget" | ||
# parse APKINDEX to find the current versions | ||
static_pkgs=$($wget/APKINDEX.tar.gz | \ | ||
tar -Oxz APKINDEX | \ | ||
awk -F: -v pkglist=$pkglist ' | ||
BEGIN { split(pkglist,pkg) } | ||
$0 != "" { f[$1] = $2 } | ||
$0 == "" { for (i in pkg) | ||
if (pkg[i] == f["P"]) | ||
print(f["P"] "-" f["V"] ".apk") }') | ||
[ "$static_pkgs" ] || return 1 | ||
|
||
mkdir -p "$rootfs" || return 1 | ||
|
||
for pkg in $static_pkgs; do | ||
echo "Downloading $pkg" | ||
$wget/$pkg | tar -xz -C "$rootfs" | ||
done | ||
|
||
# clean up .apk meta files | ||
rm -f "$rootfs"/.[A-Z]* | ||
|
||
# verify checksum of the key | ||
keyname=$(echo $rootfs/sbin/apk.static.*.pub | sed 's/.*\.SIGN\.RSA\.//') | ||
checksum=$(echo "$key_sha256sums" | grep -w "$keyname") | ||
if [ -z "$checksum" ]; then | ||
echo "ERROR: checksum is missing for $keyname" | ||
return 1 | ||
fi | ||
mkdir $rootfs/etc/apk/keys | ||
cp -a ./keys/* $rootfs/etc/apk/keys | ||
(cd $rootfs/etc/apk/keys && echo "$checksum" | sha256sum -c -) || return 1 | ||
|
||
# verify the static apk binary signature | ||
APK=$rootfs/sbin/apk.static | ||
openssl dgst -sha1 -verify $rootfs/etc/apk/keys/$keyname \ | ||
-signature "$APK.SIGN.RSA.$keyname" "$APK" || return 1 | ||
|
||
if [ "$auto_repo_dir" ]; then | ||
mirror_list=$rootfs/usr/share/alpine-mirrors/MIRRORS.txt | ||
mirror_count=$(wc -l $mirror_list | cut -d " " -f 1) | ||
random=$(hexdump -n 2 -e '/2 "%u"' /dev/urandom) | ||
repository=$(sed $(expr $random % $mirror_count + 1)\!d \ | ||
$mirror_list)$auto_repo_dir | ||
echo "Selecting mirror $repository" | ||
fi | ||
} | ||
|
||
install_alpine() { | ||
mkdir -p "$rootfs"/etc/apk || return 1 | ||
: ${keys_dir:=/etc/apk/keys} | ||
if ! [ -d "$rootfs"/etc/apk/keys ] && [ -d "$keys_dir" ]; then | ||
cp -r "$keys_dir" "$rootfs"/etc/apk/keys | ||
fi | ||
if [ -n "$repository" ]; then | ||
echo "$repository" > "$rootfs"/etc/apk/repositories | ||
else | ||
cp /etc/apk/repositories "$rootfs"/etc/apk/repositories || return 1 | ||
if [ -n "$release" ]; then | ||
sed -E -i "s:/[^/]+/([^/]+)$:/$release/\\1:" \ | ||
"$rootfs"/etc/apk/repositories | ||
fi | ||
fi | ||
opt_arch= | ||
if [ -n "$apk_arch" ]; then | ||
opt_arch="--arch $apk_arch" | ||
fi | ||
$APK add -U --initdb --root $rootfs $opt_arch "$@" alpine-base | ||
} | ||
|
||
package_image() { | ||
tar -czf rootfs.tar.gz --numeric-owner -C rootfs/ . | ||
rm -rf rootfs | ||
} | ||
|
||
die() { | ||
echo "$@" >&2 | ||
exit 1 | ||
} | ||
|
||
usage() { | ||
cat >&2 <<EOF | ||
Usage: $(basename $0) [-h|--help] [-r|--repository <url>] | ||
[-R|--release <release>] [-a|--arch <arch>] | ||
[PKG...] | ||
EOF | ||
} | ||
|
||
usage_err() { | ||
usage | ||
exit 1 | ||
} | ||
|
||
build_docker(){ | ||
echo "Building docker image" | ||
docker build -t alpine:base -f Dockerfile.builder . && rm rootfs.tar.gz | ||
docker build -t alpine:builder -f Dockerfile . | ||
} | ||
|
||
rootfs="$(pwd)/rootfs" | ||
release= | ||
arch=$(uname -m) | ||
|
||
# template requires root | ||
if [ $(id -u) -ne 0 ]; then | ||
echo "$(basename $0): must be run as root" >&2 | ||
exit 1 | ||
fi | ||
|
||
options=$(getopt -o h:r:R:a: -l help,repository:,release:,arch: -- "$@") | ||
[ $? -eq 0 ] || usage_err | ||
eval set -- "$options" | ||
|
||
while [ $# -gt 0 ]; do | ||
case "$1" in | ||
-h|--help) | ||
usage | ||
exit 0 | ||
;; | ||
-r|--repository) | ||
repository=$2 | ||
;; | ||
-R|--release) | ||
release=$2 | ||
;; | ||
-a|--arch) | ||
arch=$2 | ||
;; | ||
--) | ||
shift | ||
break;; | ||
esac | ||
shift 2 | ||
done | ||
|
||
|
||
if [ -z "$rootfs" ]; then | ||
rootfs="$(pwd)/rootfs" | ||
fi | ||
|
||
## Verify BootStrap Script | ||
checkSha=`cat .bootstrap.check` | ||
fileCheck=`openssl sha1 $(pwd)/bootstrap | awk '{print $2}'` | ||
if [ $fileCheck != $checkSha ]; then | ||
die "Failed to verify bootstap, File could have been tampered with!!!!!!" | ||
fi | ||
|
||
apk_arch=$arch | ||
|
||
case "$arch" in | ||
i[3-6]86) | ||
apk_arch=x86 | ||
;; | ||
x86) | ||
;; | ||
x86_64|"") | ||
;; | ||
arm*) | ||
apk_arch=armhf | ||
;; | ||
*) | ||
die "unsupported architecture: $arch" | ||
;; | ||
esac | ||
|
||
: ${APK:=apk} | ||
if ! which $APK >/dev/null; then | ||
get_static_apk || die "Failed to download a valid static apk" | ||
fi | ||
|
||
install_alpine "$@" || die "Failed to install rootfs" | ||
package_image || die "Failed to package image" | ||
build_docker || die "Failed to build docker container" |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can simply use the http://dl-cdn.alpinelinux.org/alpine mirror
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I pulling the mirror list from the official alpine-mirrors package. This way if we are only pulling for official mirrors, and not a dns value that might be hijacked on someones network. Otherwise your correct we could just use the cdn mirror and relay on dns doing the picking.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes no difference. Any of the mirrors could be dns hijacked or the official mirror sysadmin could go rough. What protects you from any of this to happen is the signature verification using the apk keys which you verify with the embedded sha256 sum.
In other words, as long as you trust that the builder script itself has not been modified over the wire, then you can verify that
apk
is ok andapk
will verify that the apk content is not modified, using the apk keys.I am ok to keep the current approach if you have strong feeling for it. It is good to not hard-code the mirror in case we change it in future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah it doesn't matter to me. The bootstrap script does a sha1 check on itself to make sure it hasn't been modified from outside sources :) we can swap that out for the cdn, at the top we pull in the list of mirrors from the cdn at the beginning of the process. I just found this was the best way for I work. Just figured others might be in the same boat.