Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jetlag CI stuff #511

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions ci/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
This directory is a holding area for materials which will be used to create a
Jetlag continuous integration (CI) system, to automate building and testing of
Jetlag for development and quality assurance.

Currently, there is only one file, deploy_sno.sh, which is an initial effort
at scripting the deployment of a single-node OpenShift cluster using Jetlag.
It is targetted at a Scale Lab deployment, but the techniques it uses should
be applicable across all deployments. It is currently highly opinionated, but
it should be straightforward to generalize it into a form which could be used
as a runner by Jenkins.
108 changes: 108 additions & 0 deletions ci/deploy_sno.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
#!/bin/bash -e

# User variables
BASTION=${BASTION:-"f30-h01-000-r640.rdu2.scalelab.redhat.com"}
ID_FILE=${ID_FILE:-~/.ssh/id_scalelab}
PULL_SECRET=${PULL_SECRET:-"pull_secret.txt"}
RHEL_RELEASE=${RHEL_RELEASE:-"8.9"}

# Lab & cluster infrastructure variables
LAB_NAME=${LAB_NAME:-"scalelab"}
CLOUD_NAME=${CLOUD_NAME:-"cloud02"}
CLUSTER_TYPE=${CLUSTER_TYPE:-"sno"}
SNO_NODE_COUNT=${SNO_NODE_COUNT:-"1"}
#ocp_release_image
#openshift_version
#smcipmitool_url

# Bastion node variables
#bastion_lab_interface
#bastion_controlplane_interface
#controlplane_lab_interface

# Debugging aids and other shorthand:
# - If the script is invoked with -x (the "xtrace" option), then arrange to
# pass that option to the remote execution; also, don't redirect output to
# the bit bucket -- let it show on the terminal.
# - If DEBUG is set, then echo the commands instead of executing them.
xtrace=$(set -o | sed -nE -e '/xtrace/ s/xtrace *\t*on/-x/p')
if [[ -n ${xtrace} ]]; then B=''; else B='2>/dev/null'; fi
ON_BASTION="${N} ssh -i ${ID_FILE} root@${BASTION} /bin/bash -e ${xtrace}"
N=${DEBUG:+'echo [DRY-RUN] '}
Comment on lines +30 to +31
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expand ${N} into ON_BASTION and then define N? Interesting strategy. Or did you mean ON_BASTION='${N} ...' to defer substitution until later?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those are excellent questions. I concur that the code as it stands looks suspiciously like a botch.

Unfortunately, the requirements for a "dry-run" are less than cut and...er...dried. In one conception, one would like to be able to run the script without it actually requiring or expending any resources. But, if that execution takes place only on the local node, it gets pretty boring pretty fast. A more interesting scenario involves doing a "dry run" but including the parts which execute on the bastion as well, but doing them as a dry-run. In that case, the code above does the "right thing": it actually executes the ssh command but the commands that that executes get the ${N} treatment. (However, once the mechanics of that work, then even that becomes boring...so, we should probably just tear out the whole ${N} thing altogether.)


cat <<- __EOF__
Deploying an Openshift Cluster:
bastion node: ${BASTION}
update the bastion OS version to: RHEL ${RHEL_RELEASE}
SSH ID file: ${ID_FILE}.

LAB_NAME: ${LAB_NAME}
CLOUD_NAME: ${CLOUD_NAME}
CLUSTER_TYPE: ${CLUSTER_TYPE}
SNO_NODE_COUNT: ${SNO_NODE_COUNT}

NOTE: You may be prompted by ssh-copy-id twice to enter the root password
for the bastion, once now and once after the node reboots.

__EOF__

echo -e "\nCopying local SSH key to the bastion."
${N} ssh-copy-id -i "${ID_FILE}" "root@${BASTION}" >/dev/null 2>&1

echo -e "\nChecking bastion OS version."
current_release=$(${ON_BASTION} <<< 'cat /etc/redhat-release' | ${N} awk '{ print $6 }')
if [[ "${current_release}" != "${RHEL_RELEASE}" ]]; then
${N} echo -e "\nUpdating to bastion OS from ${current_release} to ${RHEL_RELEASE}."
# shellcheck disable=SC2087
${ON_BASTION} <<- __EOF__
${N} ./update-latest-rhel-release.sh ${RHEL_RELEASE}
${N} dnf update -y
${N} reboot
__EOF__

TIMEFORMAT="elapsed %3lR; proceeding."
echo -en "\nWaiting for reboot (this may take several minutes)"
time (until ${ON_BASTION} <<< "exit 0" ${B}; do
echo Result: $?
echo -n .
sleep 10
done)
TIMEFORMAT=
fi

echo -e "\nInstalling tools, local SSH key, and Jetlag."
# shellcheck disable=SC2087
${ON_BASTION} <<- __EOF__
${N} sed -E -e 's/(.*)/\nBastion OS: \1/' /etc/redhat-release
${N} dnf install tmux git python3-pip sshpass -y
${N} rm -rf /root/.ssh/id_rsa /root/.ssh/id_rsa.pub /root/jetlag
${N} ssh-keygen -q -N "" -f /root/.ssh/id_rsa
${N} cat /root/.ssh/authorized_keys /root/.ssh/id_rsa.pub | ${N} tee /root/.ssh/authorized_keys.new >/dev/null
${N} mv -f --backup=simple --suffix=.prev /root/.ssh/authorized_keys.new /root/.ssh/authorized_keys
${N} git clone https://github.com/redhat-performance/jetlag.git && cd jetlag || exit
__EOF__

echo -e "\nCopying pull secret."
${N} scp -i "${ID_FILE}" "${PULL_SECRET}" "root@${BASTION}:jetlag/pull_secret.txt"

echo -e "\nSetting up"
# shellcheck disable=SC2087
${ON_BASTION} <<- __EOF__
${N} cd jetlag
${N} source bootstrap.sh
${N} sed -E \
-e '/lab:/ s/lab:.*/lab: ${LAB_NAME}/' \
-e '/lab_cloud:/ s/lab_cloud:.*/lab_cloud: ${CLOUD_NAME}/' \
-e '/cluster_type:/ s/cluster_type:.*/cluster_type: ${CLUSTER_TYPE}/' \
-e '/sno_node_count:/ s/sno_node_count:.*/sno_node_count: ${SNO_NODE_COUNT}/' \
ansible/vars/all.sample.yml | ${N} tee ansible/vars/all.yml
${N} ansible-playbook ansible/create-inventory.yml
${N} ansible-playbook -i ansible/inventory/cloud99.local ansible/setup-bastion.yml
__EOF__

echo -e "\nDeploying the cluster."
${ON_BASTION} <<- __EOF__
${N} cd jetlag
${N} source .ansible/bin/activate
${N} ansible-playbook -i ansible/inventory/cloud02.local ansible/sno-deploy.yml
__EOF__