Skip to content

fix(ci): Clean up GH environment so there is more disk space for workflows #1108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
peterj merged 2 commits into from
Nov 17, 2025
Merged

fix(ci): Clean up GH environment so there is more disk space for workflows #1108

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
97380dd
clean up ci env to clear up disk space for gh workflows
inFocus7 Nov 17, 2025
95573bb
remove unused directory input
inFocus7 Nov 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions .github/actions/initialize-environment/action.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: Initialize Environment

description: Initialization steps for kagent actions

runs:
using: "composite"
steps:
- name: Cancel Previous Actions
uses: styfle/cancel-workflow-action@85880fa0301c86cca9da44039ee3bb12d3bedbfa # 0.12.1
with:
access_token: ${{ github.token }}
- name: Free disk space
shell: bash
run: |
echo "Before clearing disk space:"
df -h
docker system df -v

# https://github.com/actions/runner-images/discussions/3242 github runners are bad at cleanup
echo "Removing large packages"
sudo apt-get remove -y '^dotnet-.*' || true
sudo apt-get remove -y '^llvm-.*' || true
sudo apt-get remove -y 'php.*' || true
sudo apt-get remove -y '^mongodb-.*' || true
sudo apt-get remove -y '^mysql-.*' || true
sudo apt-get remove -y azure-cli google-chrome-stable firefox powershell mono-devel libgl1-mesa-dri || true
sudo apt-get autoremove -y || true
sudo apt-get clean -y || true
echo "Done removing large packages"

# Clean up pre-installed tools
# For some reason, GHA often takes minutes (up to 20min observed) to clean up somehow.
# Presumably this is due to a slow disk?
function clean-dir() {
echo "Cleaning $1"
time sudo rm -rf "$1" || true
}
# These two were found to take very very long. Even though its large, its not worth the cost
# clean-dir /usr/local/lib/android # 8.9gb
# clean-dir $AGENT_TOOLSDIRECTORY # 5.6gb
clean-dir /usr/share/dotnet # 3.4gb
clean-dir /usr/share/swift # 3.1gb
clean-dir /usr/local/.ghcup # 6.3gb
clean-dir /usr/local/share/powershell # 1.2gb
clean-dir /usr/local/share/chromium # 600mb

# Clean up images
docker image rm node:18 || true
docker image rm node:18-alpine || true
docker image rm node:20 || true
docker image rm node:20-alpine || true
# remove the dangling images and containers
docker images | tail -n +2 | awk '$1 == "<none>" {print $3}' | xargs --no-run-if-empty docker rmi
docker ps -a | tail -n +2 | awk '$2 ~ "^[0-9a-f]+$" {print $1}' | xargs --no-run-if-empty docker rm --volumes=true

# Clean up Docker
docker system prune -f || true

echo "After clearing disk space:"
df -h
docker system df -v
2 changes: 2 additions & 0 deletions .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Initialize Environment
uses: ./.github/actions/initialize-environment
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
Expand Down
6 changes: 4 additions & 2 deletions python/samples/crewai/poem_flow/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y \
build-essential \
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/apt/lists/* \
&& apt-get clean
Copy link

Copilot AI Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] apt-get clean is redundant after rm -rf /var/lib/apt/lists/*. The rm -rf /var/lib/apt/lists/* command already removes the package lists, and apt-get clean removes downloaded package files from /var/cache/apt/archives/, which is already a minimal operation. Since the standard practice is to use rm -rf /var/lib/apt/lists/* alone in Dockerfiles for size reduction, the additional apt-get clean doesn't provide meaningful benefit and can be removed.

Suggested change
&& apt-get clean

Copilot uses AI. Check for mistakes.

# Copy project files
COPY samples samples
Expand All @@ -19,7 +20,8 @@ COPY .python-version .python-version
COPY uv.lock uv.lock

# Install dependencies
RUN uv venv && uv sync --locked --no-dev --package poem-flow
RUN uv venv && uv sync --locked --no-dev --package poem-flow \
&& uv cache clean

# Set environment variables
ENV PORT=8080
Expand Down
6 changes: 4 additions & 2 deletions python/samples/crewai/research-crew/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y \
build-essential \
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/apt/lists/* \
&& apt-get clean

Comment on lines +11 to 13
Copy link

Copilot AI Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] apt-get clean is redundant after rm -rf /var/lib/apt/lists/*. The rm -rf /var/lib/apt/lists/* command already removes the package lists, and apt-get clean removes downloaded package files from /var/cache/apt/archives/, which is already a minimal operation. Since the standard practice is to use rm -rf /var/lib/apt/lists/* alone in Dockerfiles for size reduction, the additional apt-get clean doesn't provide meaningful benefit and can be removed.

Suggested change
&& rm -rf /var/lib/apt/lists/* \
&& apt-get clean
&& rm -rf /var/lib/apt/lists/*

Copilot uses AI. Check for mistakes.
# Copy project files
COPY samples samples
Expand All @@ -19,7 +20,8 @@ COPY .python-version .python-version
COPY uv.lock uv.lock

# Install dependencies
RUN uv venv && uv sync --locked --no-dev --package research-crew
RUN uv venv && uv sync --locked --no-dev --package research-crew \
&& uv cache clean

# Set environment variables
ENV PORT=8080
Expand Down
6 changes: 4 additions & 2 deletions python/samples/langgraph/currency/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y \
build-essential \
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/apt/lists/* \
&& apt-get clean

Comment on lines +11 to 13
Copy link

Copilot AI Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] apt-get clean is redundant after rm -rf /var/lib/apt/lists/*. The rm -rf /var/lib/apt/lists/* command already removes the package lists, and apt-get clean removes downloaded package files from /var/cache/apt/archives/, which is already a minimal operation. Since the standard practice is to use rm -rf /var/lib/apt/lists/* alone in Dockerfiles for size reduction, the additional apt-get clean doesn't provide meaningful benefit and can be removed.

Suggested change
&& rm -rf /var/lib/apt/lists/* \
&& apt-get clean
&& rm -rf /var/lib/apt/lists/*

Copilot uses AI. Check for mistakes.
# Copy project files
COPY samples samples
Expand All @@ -19,7 +20,8 @@ COPY .python-version .python-version
COPY uv.lock uv.lock

# Install dependencies
RUN uv sync --locked --no-dev
RUN uv sync --locked --no-dev \
&& uv cache clean

# Set environment variables
ENV PYTHONPATH=/app
Expand Down
Loading