Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
3c14b11
#92: removed quickstart as base image and TARGET_NETWORK, as now only…
sreuland Oct 13, 2025
edd58db
#92: added callable workflow using qs pipeline from system-test
sreuland Oct 14, 2025
fef8f6b
#92: fixed cli ref url parsing for sha lookup
sreuland Oct 14, 2025
625411e
#92: fixed cli ref url parse with delimeter
sreuland Oct 14, 2025
e62f762
#92: fixed system test checkout to workspace in job
sreuland Oct 14, 2025
5a0766e
get the proper commit sha for system test git ref job parameter
sreuland Oct 14, 2025
bf2c832
use artifacts for image across jobs on forked prs
sreuland Oct 15, 2025
6ff5513
use tmp for artifact path
sreuland Oct 15, 2025
100b62c
use docker.io for image cache
sreuland Oct 15, 2025
f6efb85
add some debug output on artifact upload
sreuland Oct 15, 2025
f6f7324
add more debug output
sreuland Oct 15, 2025
2c18c7f
run quickstart first as it resets docker context, then load system te…
sreuland Oct 15, 2025
786126c
add ddebug out on startup
sreuland Oct 15, 2025
e844ce2
use add-host to reference quickstart container ports from system test…
sreuland Oct 15, 2025
e8ce9e8
#92: renamed new workflow to test-workflow
sreuland Oct 15, 2025
4355789
#92: refined registry allowed check for dockerhub
sreuland Oct 15, 2025
d83626d
#92: fix check for absent docker creds
sreuland Oct 15, 2025
9305bc3
fixed job filter to check for skipped due to image cache or success o…
sreuland Oct 16, 2025
68bf397
#92: make rpc version an input variable and all input component names…
sreuland Oct 16, 2025
c71f463
#92: allow cli crate version input variable on workflow
sreuland Oct 16, 2025
77445f4
#92: consistent js stellar sdk gh input var names
sreuland Oct 16, 2025
ab61d6d
#92: consistent input variable names for soroban examples repo/ref
sreuland Oct 16, 2025
f396c7a
#92: fixed cli repo docker git url for context format
sreuland Oct 16, 2025
6f8fddd
#92: use gha caches for docker build layer caching
sreuland Oct 24, 2025
ac79f9f
#92: fix gha cache verstion syntax on step
sreuland Oct 24, 2025
60f098c
#92: remove unnecessary cache key creation
sreuland Oct 25, 2025
405dab9
#92: use artifacts instead of cache for interim docker image for syst…
sreuland Oct 27, 2025
34febf8
#92: specify system-test repo for gha cache ownership
sreuland Nov 4, 2025
202124c
#92: enable plain logging on docker build to see caching activity
sreuland Nov 4, 2025
eb1c902
#92: dont' use repository param in gha caching, not effective
sreuland Nov 4, 2025
ada2d32
#92: dont' use gha docker buildx action
sreuland Nov 5, 2025
f2edf87
#92: need --load for docker buildx to save to local repo
sreuland Nov 5, 2025
b19e8bd
#92: specify docker driver for buildx
sreuland Nov 5, 2025
6509db3
#92: use gha cache on ci test build
sreuland Nov 5, 2025
6df0d5c
#92: added debug on buildx
sreuland Nov 5, 2025
272cc86
#92: corrected debug on buildx
sreuland Nov 5, 2025
ff8a819
#92: don't use equals on the buildx cache params
sreuland Nov 5, 2025
9ec6778
#92: don't use quotes on makefile variable
sreuland Nov 5, 2025
8b77666
#92: use new horizon and friendbot refs to quickstart flow
sreuland Nov 15, 2025
68bbc32
#92: include crazy-max gha action to assist docker gha caching
sreuland Nov 17, 2025
64ad22d
#92: use docker container driver for buildx
sreuland Nov 17, 2025
0357394
#92: fix getEvents endLedger requirement for latest v24 js sdk
sreuland Nov 17, 2025
de79a02
#92: use docker-container driver for cache exports, oci export for cl…
sreuland Nov 17, 2025
25b5aec
#92: use docker-container for driver with buildx on callable workflow
sreuland Nov 17, 2025
3ef80d1
#92: focus on local buildx cache using incremental action cache keys
sreuland Nov 18, 2025
a3e28fc
#92: trying to use volume mount from buildkit container to runner hos…
sreuland Nov 18, 2025
3d77671
#92: use driver-opt param for vol mount
sreuland Nov 18, 2025
5c4485d
#92: run buildkitd container directly to specify vol mount to gha run…
sreuland Nov 18, 2025
de1f98a
#92: fix unix socket name, don't use setup-buildx action, not needed
sreuland Nov 18, 2025
e0eb6e5
#92: fix vol mounts for socket path on buildkitd run
sreuland Nov 18, 2025
f91366a
#92: print logs of buildkit container start error
sreuland Nov 18, 2025
3fdc993
#92: use tcp port for buildkitd instead of unix socket to gha runner
sreuland Nov 18, 2025
6b90e1f
#92: revert back to image file cache pattern for ci build
sreuland Nov 18, 2025
1923043
#92: revert back to doing cache per image on callable workflow
sreuland Nov 18, 2025
92ba87e
#92: pass the compressed image tar from build to run job
sreuland Nov 18, 2025
5c00664
#92: added console output for compression stats on image files
sreuland Nov 18, 2025
4316c54
#92 use lower compression ration on image files for faster time
sreuland Nov 18, 2025
4a6862e
#92 console verbose logging on compression status
sreuland Nov 18, 2025
16aa20c
#92: include the git ref for system-test version in the hashed system…
sreuland Nov 18, 2025
df2e9f4
#92: resolve component refs to shas for cache keys
sreuland Nov 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 61 additions & 47 deletions .github/workflows/Dockerfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on:
branches:
- master
tags:
- '*'
- "*"
pull_request:
release:
types: [published]
Expand All @@ -19,55 +19,69 @@ env:
# the only time system test image needs to be defined for a push is when it's a non 'v' tag creation
PUSH_ENABLED: ${{ secrets.DOCKERHUB_TOKEN && github.ref_type == 'tag' && startsWith(github.ref_name, 'v') != true && 'true' || 'false' }}
IMAGE: ${{ format('{0}/{1}:{2}', secrets.DOCKERHUB_TOKEN && 'docker.io' || 'ghcr.io', github.repository, github.ref_type == 'tag' && github.ref_name || github.event.ref || 'latest') }}
CORE_GIT_REF: https://github.com/stellar/stellar-core.git#master
STELLAR_RPC_GIT_REF: https://github.com/stellar/stellar-rpc.git#protocol-23
RUST_TOOLCHAIN_VERSION: stable
SOROBAN_CLI_GIT_REF: https://github.com/stellar/soroban-tools.git#main
QUICKSTART_GIT_REF: https://github.com/stellar/quickstart.git#main
# leaving sdk npm version blank defaults to whatever npm has for latest version
# rather than build from git source, which is fine for ci test build
JS_STELLAR_SDK_NPM_VERSION:
RS_XDR_GIT_REPO: https://github.com/stellar/rs-stellar-xdr
RS_XDR_GIT_REF: main
jobs:
complete:
if: always()
needs: [build]
runs-on: ubuntu-latest
steps:
- if: contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')
run: exit 1
- if: contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')
run: exit 1

build:
runs-on: ubuntu-latest-16-cores
outputs:
push_enabled: ${{ env.PUSH_ENABLED }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ env.HEAD_SHA }}
- name: Build System Test Image
run: |
make CORE_GIT_REF=${{ env.CORE_GIT_REF }} \
CORE_COMPILE_CONFIGURE_FLAGS="${{ env.CORE_COMPILE_CONFIGURE_FLAGS }}" \
STELLAR_RPC_GIT_REF=${{ env.STELLAR_RPC_GIT_REF }} \
RUST_TOOLCHAIN_VERSION=${{ env.RUST_TOOLCHAIN_VERSION }} \
SOROBAN_CLI_GIT_REF=${{ env.SOROBAN_CLI_GIT_REF }} \
SYSTEM_TEST_IMAGE=${{ env.PUSH_ENABLED == 'true' && env.IMAGE || 'stellar/system-test:dev' }} \
JS_STELLAR_SDK_NPM_VERSION=${{ env.JS_STELLAR_SDK_NPM_VERSION }} \
RS_XDR_GIT_REPO=${{ env.RS_XDR_GIT_REPO }} \
RS_XDR_GIT_REF=${{ env.RS_XDR_GIT_REF }} \
QUICKSTART_GIT_REF=${{ env.QUICKSTART_GIT_REF }} build;
- if: ${{ env.PUSH_ENABLED == 'true' }}
name: Save Docker Image to file
run: |
docker save ${{ env.IMAGE }} -o /tmp/image;
- if: ${{ env.PUSH_ENABLED == 'true' }}
name: Upload System Test Image
uses: actions/upload-artifact@v4
with:
name: image-Dockerfile
path: /tmp/image
- uses: actions/checkout@v4
with:
ref: ${{ env.HEAD_SHA }}
- name: Restore image cache
id: image-cache
uses: actions/cache/restore@v4
with:
path: /tmp/image.tar.zst
key: systemtest-image-${{ runner.os }}-${{ github.sha }}

- name: Load cached Docker image
if: steps.image-cache.outputs.cache-hit == 'true'
run: |
zstd -d -c /tmp/image.tar.zst | docker load

- name: Build System Test Image
if: steps.image-cache.outputs.cache-hit != 'true'
run: |
make SYSTEM_TEST_IMAGE=${{ env.PUSH_ENABLED == 'true' && env.IMAGE || 'stellar/system-test:dev' }} \
JS_STELLAR_SDK_NPM_VERSION=${{ env.JS_STELLAR_SDK_NPM_VERSION }} \
build;

- name: Save Docker Image to file
if: steps.image-cache.outputs.cache-hit != 'true'
run: |
docker save ${{ env.PUSH_ENABLED == 'true' && env.IMAGE || 'stellar/system-test:dev' }} | zstd -19 -o /tmp/image.tar.zst

- name: Save image cache
if: steps.image-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v4
with:
path: /tmp/image.tar.zst
key: systemtest-image-${{ runner.os }}-${{ github.sha }}

- if: ${{ env.PUSH_ENABLED == 'true' }}
name: Decompress image for upload
run: |
zstd -d /tmp/image.tar.zst -o /tmp/image.tar

- if: ${{ env.PUSH_ENABLED == 'true' }}
name: Upload System Test Image
uses: actions/upload-artifact@v4
with:
name: image-Dockerfile
path: /tmp/image.tar

push:
# Only push non 'vX.Y.Z' tags as pre-built system test image with versions compiled into image.
Expand All @@ -80,16 +94,16 @@ jobs:
runs-on: ubuntu-latest
if: needs.build.outputs.push_enabled == 'true'
steps:
- uses: actions/checkout@v4
with:
ref: ${{ env.HEAD_SHA }}
- id: push
uses: ./.github/actions/push
with:
head_sha: ${{ env.HEAD_SHA }}
artifact_name: image-Dockerfile
artifact_image_file: image
image: ${{ env.IMAGE }}
registry: ${{ secrets.DOCKERHUB_TOKEN && 'docker.io' || 'ghcr.io' }}
username: ${{ secrets.DOCKERHUB_USERNAME || github.actor }}
password: ${{ secrets.DOCKERHUB_TOKEN || github.token }}
- uses: actions/checkout@v4
with:
ref: ${{ env.HEAD_SHA }}
- id: push
uses: ./.github/actions/push
with:
head_sha: ${{ env.HEAD_SHA }}
artifact_name: image-Dockerfile
artifact_image_file: image.tar
image: ${{ env.IMAGE }}
registry: ${{ secrets.DOCKERHUB_TOKEN && 'docker.io' || 'ghcr.io' }}
username: ${{ secrets.DOCKERHUB_USERNAME || github.actor }}
password: ${{ secrets.DOCKERHUB_TOKEN || github.token }}
Loading