Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

amend linux cleanup script with full minikube, docker and kvm cleanup #9726

Merged
merged 4 commits into from
Dec 11, 2020
Merged

amend linux cleanup script with full minikube, docker and kvm cleanup #9726

Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1b52103
amend linux cleanup script with full minikube, docker and kvm cleanup
prezha Nov 17, 2020
973640c
ensure the script will not fail; keep cached docker images
prezha Nov 17, 2020
cefceeb
give minikube a chance to clenup itself
prezha Nov 17, 2020
e442cde
restrict cleanup only to jenkins and root user
prezha Dec 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 78 additions & 6 deletions hack/jenkins/cron/cleanup_and_reboot_Linux.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash

# Copyright 2019 The Kubernetes Authors All rights reserved.
# Copyright 2020 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -36,11 +36,83 @@ logger "cleanup_and_reboot is happening!"
# kill jenkins to avoid an incoming request
killall java

# clean minikube left overs
echo -e "\ncleanup minikube..."
for user in $(lslogins --user-accs --noheadings --output=USER); do
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can assume only jenkins and root (for the none driver) as users here here instead of using the for loop across all users

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, i've hardcoded only jenkins and root user as targets for the cleanup

minikube="$(sudo su - ${user} -c 'command -v minikube')"
if [ ! -x "${minikube}" ]; then
minikube="/tmp/minikube"
curl -sfL https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 -o "${minikube}" && chmod +x "${minikube}" || true
fi
if [ -x "${minikube}" ]; then
if sudo su - "${user}" -c "${minikube} delete --all --purge" >/dev/null 2>&1; then
echo "successfully cleaned up minikube for ${user} user using ${minikube}"
fi
fi
sudo killall --user "${user}" minikube >/dev/null 2>&1 || true
done

# clean docker left overs
docker rm -f -v $(docker ps -aq) >/dev/null 2>&1 || true
docker volume prune -f || true
docker volume ls || true
docker system df || true
echo -e "\ncleanup docker..."
docker kill $(docker ps -aq) >/dev/null 2>&1 || true
docker system prune --volumes --force || true

# clean KVM left overs
echo -e "\ncleanup kvm..."
overview() {
echo -e "\n - KVM domains:"
sudo virsh list --all || true
echo " - KVM pools:"
sudo virsh pool-list --all || true
echo " - KVM networks:"
sudo virsh net-list --all || true
echo " - host networks:"
sudo ip link show || true
}
echo -e "\nbefore the cleanup:"
overview
for DOM in $( sudo virsh list --all --name ); do
if sudo virsh destroy "${DOM}"; then
if sudo virsh undefine "${DOM}"; then
echo "successfully deleted KVM domain:" "${DOM}"
continue
fi
echo "unable to delete KVM domain:" "${DOM}"
fi
done
#for POOL in $( sudo virsh pool-list --all --name ); do # better, but flag '--name' is not supported for 'virsh pool-list' command on older libvirt versions
for POOL in $( sudo virsh pool-list --all | awk 'NR>2 {print $1}' ); do
for VOL in $( sudo virsh vol-list "${POOL}" ); do
if sudo virsh vol-delete --pool "${POOL}" "${VOLUME}"; then # flag '--delete-snapshots': "delete snapshots associated with volume (must be supported by storage driver)"
echo "successfully deleted KVM pool/volume:" "${POOL}"/"${VOL}"
continue
fi
echo "unable to delete KVM pool/volume:" "${POOL}"/"${VOL}"
done
done
for NET in $( sudo virsh net-list --all --name ); do
if [ "${NET}" != "default" ]; then
if sudo virsh net-destroy "${NET}"; then
if sudo virsh net-undefine "${NET}"; then
echo "successfully deleted KVM network" "${NET}"
continue
fi
fi
echo "unable to delete KVM network" "${NET}"
fi
done
# DEFAULT_BRIDGE is a bridge connected to the 'default' KVM network
DEFAULT_BRIDGE=$( sudo virsh net-info default | awk '{ if ($1 == "Bridge:") print $2 }' )
echo "bridge connected to the 'default' KVM network to leave alone:" "${DEFAULT_BRIDGE}"
for VIF in $( sudo ip link show | awk -v defvbr="${DEFAULT_BRIDGE}.*" -F': ' '$2 !~ defvbr { if ($2 ~ /virbr.*/ || $2 ~ /vnet.*/) print $2 }' ); do
if sudo ip link delete "${VIF}"; then
echo "successfully deleted KVM interface" "${VIF}"
continue
fi
echo "unable to delete KVM interface" "${VIF}"
done
echo -e "\nafter the cleanup:"
overview

# Linux-specific cleanup

Expand All @@ -51,4 +123,4 @@ systemctl list-unit-files --state=enabled \
| xargs systemctl disable

# update and reboot
apt update -y && apt upgrade -y && reboot
apt update -y && apt upgrade -y && apt-get autoclean && reboot