Skip to content

Commit de4e7e1

Browse files
PastaPastaPastaPastaPastaPasta
committed
Merge #6171: feat: implement GitHub Actions based starter CI
acf1315 Merge bitcoin#25091: test: Remove extended lint (cppcheck) (MacroFake) 4dbdecd refactor: rename builder-image -> build-image and builder as image name to dashcore-ci-runner (pasta) ed8ffa7 feat: have cppcheck linter respect CACHE_DIR env variable (pasta) d1addb2 fix: change fallback download path to be an s3 link which includes a few packages (pasta) 35c7670 feat: implement basic Github Actions based CI, which reuses underlying logic from GitLab CI (pasta) Pull request description: ## Issue being fixed or feature implemented We currently rely on GitLab for running CI, and while it has worked quite well, I am worried about having all of our eggs in one basket! As such, I've long wanted to explore implemeenting Github Actions based CI, but was too lazy! Well I finally spent some 60 commits trying to figure everything out, and this PR is the result of it. As a result, we will now have two semi-redunant CI systems, the primary one on GitLab, and this one on Github Actions. Currently the GA based CI will only do one host, and does linting. Be aware this GA based CI does not actually run the tests, it does build depends, src and run linters on 1 host. In the future, we should expand it from simply arm32 builds to having feature parity to GitLab. While it appears the GA default runners are a bit slower than what we have on GitLab, there's a big difference, the GA runners are free :D If we decide to make the GA based CI primary, we'll probably want to setup some custom runners to have improved build speeds. Even still, a heavily cached build doing all linters took around 5 minutes if I recall correctly. Without caches I think it took maybe an hour, so defenitely not bad. ## What was done? See the individual commits, they're pretty self explanatory ## How Has This Been Tested? Lots of CI runs on my prior branch :) CI should run on this PR, and we should see how long it'll take w/o cache :D ## Breaking Changes N/A - CI only ## Checklist: _Go over all the following points, and put an `x` in all the boxes that apply._ - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have added or updated relevant unit/integration/functional/e2e tests - [x] I have made corresponding changes to the documentation - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ Top commit has no ACKs. Tree-SHA512: fef41d1b73fdeac29e5096ffa7fa26660efc44ac274376d5880b5fabf6b0e760aeda4a4ae9c24656ee0e3adb760459f0d45b955cefffc9d0eeb5384afbc9d473
2 parents efe4c2d + acf1315 commit de4e7e1

File tree

6 files changed

+184
-116
lines changed

6 files changed

+184
-116
lines changed

‎.github/workflows/build.yml‎

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
pull_request_target:
6+
7+
permissions:
8+
contents: read
9+
packages: write
10+
11+
env:
12+
DOCKER_DRIVER: overlay2
13+
FAST_MODE: false
14+
15+
jobs:
16+
build-image:
17+
name: Build Image
18+
runs-on: ubuntu-20.04
19+
outputs:
20+
image-tag: ${{ steps.prepare.outputs.image-tag }}
21+
repo-name: ${{ steps.prepare.outputs.repo-name }}
22+
steps:
23+
- name: Checkout code
24+
uses: actions/checkout@v4
25+
26+
- name: Prepare
27+
id: prepare
28+
run: |
29+
BRANCH_NAME=$(echo "${GITHUB_REF##*/}" | tr '[:upper:]' '[:lower:]')
30+
REPO_NAME=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]')
31+
echo "::set-output name=image-tag::${BRANCH_NAME}"
32+
echo "::set-output name=repo-name::${REPO_NAME}"
33+
34+
- name: Set up Docker Buildx
35+
uses: docker/setup-buildx-action@v3
36+
37+
- name: Login to GitHub Container Registry
38+
uses: docker/login-action@v3
39+
with:
40+
registry: ghcr.io
41+
username: ${{ github.actor }}
42+
password: ${{ secrets.GITHUB_TOKEN }}
43+
44+
- name: Build and push Docker image
45+
uses: docker/build-push-action@v6
46+
with:
47+
context: ./contrib/containers/ci
48+
file: ./contrib/containers/ci/Dockerfile
49+
push: true
50+
tags: |
51+
ghcr.io/${{ steps.prepare.outputs.repo-name }}/dashcore-ci-runner:${{ steps.prepare.outputs.image-tag }}
52+
ghcr.io/${{ steps.prepare.outputs.repo-name }}/dashcore-ci-runner:latest
53+
cache-from: type=registry,ref=ghcr.io/${{ steps.prepare.outputs.repo-name }}/dashcore-ci-runner:latest
54+
cache-to: type=inline
55+
56+
build-depends:
57+
name: Build Dependencies
58+
needs: build-image
59+
runs-on: ubuntu-20.04
60+
strategy:
61+
matrix:
62+
include:
63+
- build_target: arm-linux
64+
host: arm-linux-gnueabihf
65+
container:
66+
image: ghcr.io/${{ needs.build-image.outputs.repo-name }}/dashcore-ci-runner:${{ needs.build-image.outputs.image-tag }}
67+
options: --user root
68+
steps:
69+
- name: Checkout code
70+
uses: actions/checkout@v4
71+
72+
- name: Cache dependencies
73+
uses: actions/cache@v4
74+
with:
75+
path: |
76+
depends/built
77+
depends/${{ matrix.host }}
78+
depends/sdk-sources
79+
# We don't care about no specific key as depends system will handle that for us
80+
key: ${{ runner.os }}-depends-${{ matrix.host }}-${{ github.sha }}
81+
restore-keys: |
82+
${{ runner.os }}-depends-${{ matrix.host }}-${{ github.sha }}
83+
${{ runner.os }}-depends-${{ matrix.host }}
84+
${{ runner.os }}-depends
85+
86+
- name: Build dependencies
87+
run: make -j$(nproc) -C depends HOST=${{ matrix.host }}
88+
89+
build:
90+
name: Build
91+
needs: [build-image, build-depends]
92+
runs-on: ubuntu-20.04
93+
strategy:
94+
matrix:
95+
include:
96+
- build_target: arm-linux
97+
host: arm-linux-gnueabihf
98+
dep_opts: DEBUG=1
99+
container:
100+
image: ghcr.io/${{ needs.build-image.outputs.repo-name }}/dashcore-ci-runner:${{ needs.build-image.outputs.image-tag }}
101+
options: --user root
102+
steps:
103+
- name: Checkout code
104+
uses: actions/checkout@v4
105+
106+
- name: Restore Cache dependencies
107+
uses: actions/cache/restore@v4
108+
with:
109+
path: |
110+
depends/built
111+
depends/${{ matrix.host }}
112+
depends/sdk-sources
113+
# We don't care about no specific key as depends system will handle that for us
114+
key: ${{ runner.os }}-depends-${{ matrix.host }}-${{ github.sha }}
115+
restore-keys: |
116+
${{ runner.os }}-depends-${{ matrix.host }}-${{ github.sha }}
117+
${{ runner.os }}-depends-${{ matrix.host }}
118+
${{ runner.os }}-depends
119+
120+
- name: CCache
121+
uses: actions/cache@v4
122+
with:
123+
path: |
124+
/cache
125+
key: ${{ runner.os }}-${{ matrix.host }}-${{ github.sha }}
126+
restore-keys: |
127+
${{ runner.os }}-${{ matrix.host }}-${{ github.sha }}
128+
${{ runner.os }}-${{ matrix.host }}
129+
${{ runner.os }}
130+
131+
- name: Build source and run tests
132+
run: |
133+
git config --global --add safe.directory "$PWD"
134+
CCACHE_SIZE="400M"
135+
CACHE_DIR="/cache"
136+
mkdir /output
137+
BASE_OUTDIR="/output"
138+
BUILD_TARGET="${{ matrix.build_target }}"
139+
source ./ci/dash/matrix.sh
140+
./ci/dash/build_src.sh
141+
./ci/dash/test_unittests.sh
142+
shell: bash
143+
144+
- name: Upload build artifacts
145+
uses: actions/upload-artifact@v4
146+
with:
147+
name: build-artifacts
148+
path: |
149+
/output
150+
151+
152+
# Come back to this later and implement tests :)
153+
# test:
154+
# name: Test
155+
# needs: [build-image, build]
156+
# runs-on: ubuntu-20.04
157+
# container:
158+
# image: ghcr.io/${{ needs.build-image.outputs.repo-name }}/dashcore-ci-runner:${{ needs.build-image.outputs.image-tag }}
159+
# options: --user root
160+
# steps:
161+
# - name: Checkout code
162+
# uses: actions/checkout@v4
163+
#
164+
# - name: Download build artifacts
165+
# uses: actions/download-artifact@v4
166+
# with:
167+
# name: build-artifacts
168+
# path: src/
169+
#
170+
## - name: Setup environment
171+
## run: |
172+
## echo "BUILD_TARGET=${{ needs.build.matrix.build_target }}"
173+
## source ./ci/dash/matrix.sh
174+
#
175+
# - name: Run integration tests
176+
# run: ./ci/dash/test_integrationtests.sh --extended --exclude feature_pruning,feature_dbcrash

‎ci/dash/build_src.sh‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,6 @@ if [ "$CHECK_DOC" = 1 ]; then
2626
#test/lint/check-doc.py
2727
# Run all linters
2828
test/lint/lint-all.sh
29-
test/lint/extended-lint-all.sh
3029
fi
3130

3231
ccache --zero-stats --max-size=$CCACHE_SIZE

‎depends/Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ NO_ZMQ ?=
4141
NO_UPNP ?=
4242
NO_NATPMP ?=
4343
MULTIPROCESS ?=
44-
FALLBACK_DOWNLOAD_PATH ?= https://bitcoincore.org/depends-sources
44+
FALLBACK_DOWNLOAD_PATH ?= http://dash-depends-sources.s3-website-us-west-2.amazonaws.com
4545

4646
BUILD = $(shell ./config.guess)
4747
HOST ?= $(BUILD)

‎test/lint/extended-lint-all.sh‎

Lines changed: 0 additions & 26 deletions
This file was deleted.

‎test/lint/extended-lint-cppcheck.sh‎

Lines changed: 0 additions & 86 deletions
This file was deleted.

‎test/lint/lint-cppcheck-dash.sh‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -80,10 +80,15 @@ ENABLED_CHECKS_REGEXP=$(join_array "|" "${ENABLED_CHECKS[@]}")
8080
IGNORED_WARNINGS_REGEXP=$(join_array "|" "${IGNORED_WARNINGS[@]}")
8181
FILES_REGEXP=$(join_array "|" "${FILES[@]}")
8282
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
83-
CPPCHECK_DIR=$SCRIPT_DIR/.cppcheck/
83+
# Check if CACHE_DIR is set and non-empty, otherwise use default .cppcheck/ directory
84+
if [[ -n "$CACHE_DIR" ]]; then
85+
CPPCHECK_DIR=$CACHE_DIR/cppcheck/
86+
else
87+
CPPCHECK_DIR=$SCRIPT_DIR/.cppcheck/
88+
fi
8489
if [ ! -d $CPPCHECK_DIR ]
8590
then
86-
mkdir $CPPCHECK_DIR
91+
mkdir -p $CPPCHECK_DIR
8792
fi
8893
WARNINGS=$(echo "${FILES}" | \
8994
xargs cppcheck --enable=all --inline-suppr --suppress=missingIncludeSystem --cppcheck-build-dir=$CPPCHECK_DIR -j "$(getconf _NPROCESSORS_ONLN)" --language=c++ --std=c++17 --template=gcc -D__cplusplus -DENABLE_WALLET -DCLIENT_VERSION_BUILD -DCLIENT_VERSION_IS_RELEASE -DCLIENT_VERSION_MAJOR -DCLIENT_VERSION_MINOR -DCOPYRIGHT_YEAR -DDEBUG -DUSE_EPOLL -DCHAR_BIT=8 -I src/ -q 2>&1 | sort -u | \

0 commit comments

Comments
 (0)