Skip to content

Commit

Permalink
ci: updated ci pipelines for security and best practices (hashgraph#2535
Browse files Browse the repository at this point in the history
)

* Updated to use self-hosted runners

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>

* Updated workflows for various pinned actions

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>

* Updated workflows to pin commits to actions

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>

* Added hardened runner step

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>

* Update DAPP to make `make` available

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>

* Updated workflows to get python back in

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>

* Added setup helm to charts.yml

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>

* Forgot to add setup-helm action to install in charts

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>

* Updated workflows to correct failures in runners

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>

* Updated download-artifact version

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>

* Updated timeout on acceptance-workflow:run-acceptance-tests step to 30 minutes

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>

---------

Signed-off-by: Roger Barker <roger.barker@swirldslabs.com>
  • Loading branch information
rbarkerSL authored May 28, 2024
1 parent 53e5e4b commit e8b8359
Show file tree
Hide file tree
Showing 18 changed files with 248 additions and 91 deletions.
20 changes: 20 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,23 @@
/k6/ @AlfredoG87 @hashgraph/hedera-smart-contracts
/tools/ @georgi-l95 @Ivo-Yankov @hashgraph/hedera-smart-contracts

########################
##### Core Files ######
#########################

# NOTE: Must be placed last to ensure enforcement over all other rules

# Protection Rules for Github Configuration Files and Actions Workflows
/.github/ @hashgraph/release-engineering @hashgraph/release-engineering-managers
/.github/workflows/ @hashgraph/release-engineering @hashgraph/release-engineering-managers @AlfredoG87 @ebadiere @Nana-EC @hashgraph/hedera-smart-contracts

# Self-protection for root CODEOWNERS files (this file should not exist and should definitely require approval)
/CODEOWNERS @hashgraph/release-engineering @hashgraph/release-engineering-managers

# Protect the repository root files
/README.md @hashgraph/release-engineering @hashgraph/release-engineering-managers @AlfredoG87 @ebadiere @Nana-EC @hashgraph/hedera-smart-contracts
**/LICENSE @hashgraph/release-engineering @hashgraph/release-engineering-managers

# Git Ignore definitions
**/.gitignore @hashgraph/release-engineering @hashgraph/release-engineering-managers @AlfredoG87 @ebadiere @Nana-EC @hashgraph/hedera-smart-contracts
**/.gitignore.* @hashgraph/release-engineering @hashgraph/release-engineering-managers @AlfredoG87 @ebadiere @Nana-EC @hashgraph/hedera-smart-contracts
11 changes: 8 additions & 3 deletions .github/workflows/acceptance-public.yml
Original file line number Diff line number Diff line change
Expand Up @@ -158,15 +158,20 @@ jobs:
- websocket-batch-2
- websocket-batch-3

runs-on: ubuntu-latest
runs-on: [self-hosted, Linux, medium, ephemeral]
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Download Test Reports
uses: actions/download-artifact@v3
uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # v3.0.2
with:
name: Test Results

- name: Publish Test Report
uses: actionite/publish-unit-test-result-action@v2
uses: actionite/publish-unit-test-result-action@1e01e49081c6c4073913aa4b7980fa83e709f322 # v2.3.0
with:
check_name: Test Results
json_thousands_separator: ','
Expand Down
17 changes: 11 additions & 6 deletions .github/workflows/acceptance-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,21 @@ jobs:
contents: write
# issues: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Setup node
uses: actions/setup-node@v3
uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
with:
node-version: 18

- name: Install make
run: sudo apt-get update; sudo apt-get install build-essential -y

- name: Checkout repo
uses: actions/checkout@v3
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6

- name: Install packages
run: npm ci
Expand Down Expand Up @@ -77,24 +82,24 @@ jobs:
run: docker stop json-rpc-relay json-rpc-relay-ws

- name: Run acceptance tests
uses: nick-fields/retry@v2
uses: nick-fields/retry@7152eba30c6575329ac0576536151aca5a72780e # v3.0.0
with:
max_attempts: 3
timeout_minutes: 20
timeout_minutes: 30
command: npm run acceptancetest:${{ inputs.testfilter }}
env:
TEST_WS_SERVER: ${{ inputs.test_ws_server }}
SUBSCRIPTIONS_ENABLED: ${{ inputs.test_ws_server }}

- name: Upload Test Results
if: always()
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
with:
name: Test Results
path: test-*.xml

- name: Publish Test Report
uses: actionite/publish-unit-test-result-action@v2
uses: actionite/publish-unit-test-result-action@1e01e49081c6c4073913aa4b7980fa83e709f322 # v2.3.0
if: ${{ !cancelled() }}
with:
check_run_disabled: true
Expand Down
11 changes: 8 additions & 3 deletions .github/workflows/acceptance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,15 +125,20 @@ jobs:
- websocket-batch-3
- cacheservice

runs-on: ubuntu-latest
runs-on: [self-hosted, Linux, medium, ephemeral]
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Download Test Reports
uses: actions/download-artifact@v3
uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # v3.0.2
with:
name: Test Results

- name: Publish Test Report
uses: actionite/publish-unit-test-result-action@v2
uses: actionite/publish-unit-test-result-action@1e01e49081c6c4073913aa4b7980fa83e709f322 # v2.3.0
with:
check_name: Acceptance Tests
check_run_disabled: true
Expand Down
47 changes: 38 additions & 9 deletions .github/workflows/charts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,26 +13,55 @@ concurrency:

jobs:
lint:
runs-on: ubuntu-latest
runs-on: [self-hosted, Linux, medium, ephemeral]
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6

- name: Setup Python
uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
with:
python-version: "3.11"

- name: Setup Helm
uses: Azure/setup-helm@fe7b79cd5ee1e45176fcad797de68ecaf3ca4814 # v4.2.0

- name: Install ct
uses: helm/chart-testing-action@v2.6.1
uses: helm/chart-testing-action@e6669bcd63d7cb57cb4380c33043eebe5d111992 # v2.6.1

- name: Run lint
run: ct lint --config .github/ct.yaml --all

install:
runs-on: ubuntu-latest
runs-on: [self-hosted, Linux, medium, ephemeral]
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
fetch-depth: 0
submodules: 'false'

- name: Setup Python
uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
with:
python-version: "3.11"

- name: Setup Helm
uses: Azure/setup-helm@fe7b79cd5ee1e45176fcad797de68ecaf3ca4814 # v4.2.0

- name: Setup kubectl
uses: Azure/setup-kubectl@3e0aec4d80787158d308d7b364cb1b702e7feb7f # v4.0.0

- name: Install k3d
run: curl --retry 3 -fsL https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash

Expand All @@ -41,15 +70,15 @@ jobs:
timeout-minutes: 3

- name: Set up Docker Qemu
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@68827325e0b33c7199eb31dd4e31fbe9023e06e3 # v3.0.0

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@d70bba72b1f3fd22344832f00baa16ece964efeb # v3.3.0
with:
driver-opts: network=host

- name: Build and push images
uses: docker/build-push-action@v4
uses: docker/build-push-action@2cdde995de11925a030ce8070c3d77a52ffcf1c0 # v5.3.0
with:
cache-from: type=gha
cache-to: type=gha,mode=max
Expand All @@ -59,7 +88,7 @@ jobs:
tags: localhost:5001/${{ github.repository }}:test

- name: Install ct
uses: helm/chart-testing-action@v2.6.1
uses: helm/chart-testing-action@e6669bcd63d7cb57cb4380c33043eebe5d111992 # v2.6.1

- name: Install chart
run: ct install --helm-extra-args="--timeout 10m" --all
21 changes: 15 additions & 6 deletions .github/workflows/dapp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,20 +14,29 @@ concurrency:
jobs:
setup-local-hedera:
name: Dapp Tests
runs-on: ubuntu-latest
runs-on: [self-hosted, Linux, medium, ephemeral]
timeout-minutes: 35 # Set to 35 minutes for now
permissions:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Checkout repo
uses: actions/checkout@v3
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6

- name: Setup node
uses: actions/setup-node@v3
uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
with:
node-version: 18.13.0
cache: "npm"
cache-dependency-path: "**/package-lock.json"

- name: Install build tools
run: sudo apt-get update && sudo apt-get install -y make gcc g++

- name: Install packages
run: npm ci

Expand All @@ -54,7 +63,7 @@ jobs:

- name: Dump relay logs
if: ${{ always() && !cancelled() }}
uses: jwalton/gh-docker-logs@v2
uses: jwalton/gh-docker-logs@2741064ab9d7af54b0b1ffb6076cf64c16f0220e # v2.2.2
with:
dest: './logs'

Expand All @@ -64,13 +73,13 @@ jobs:

- name: Upload logs to GitHub
if: ${{ always() && !cancelled() }}
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
with:
name: logs.tgz
path: ./logs.tgz

- name: Publish Reports
uses: mikepenz/action-junit-report@v3
uses: mikepenz/action-junit-report@9379f0ccddcab154835d4e2487555ee79614fe95 # v4.2.1
if: ${{ always() && !cancelled() }}
with:
check_name: Dapp Test Report
Expand Down
16 changes: 12 additions & 4 deletions .github/workflows/dev-tool-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,25 @@ on:

jobs:
dev-tool-workflow:
runs-on: ubuntu-latest
runs-on: [self-hosted, Linux, medium, ephemeral]
permissions:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Install build tools
run: sudo apt-get update && sudo apt-get install -y make gcc g++

- name: Setup node
uses: actions/setup-node@v3
uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
with:
node-version: 18

- name: Checkout repo
uses: actions/checkout@v3
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6

- name: Install packages
run: npm ci
Expand All @@ -49,7 +57,7 @@ jobs:
timeout-minutes: 8

- name: Run the tests
uses: nick-fields/retry@v2
uses: nick-fields/retry@7152eba30c6575329ac0576536151aca5a72780e # v3.0.0
with:
max_attempts: 10
timeout_minutes: 10
Expand Down
9 changes: 7 additions & 2 deletions .github/workflows/flow-pr-title-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,16 @@ concurrency:
jobs:
title-check:
name: Title Check
runs-on: ubuntu-latest
runs-on: [self-hosted, Linux, medium, ephemeral]
permissions:
statuses: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Check PR Title
uses: aslafy-z/conventional-pr-title-action@v3
uses: step-security/conventional-pr-title-action@0eae74515f5a79f8773fa04142dd746df76666ac # v1.0.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
11 changes: 8 additions & 3 deletions .github/workflows/foundry.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,19 @@ jobs:
fail-fast: true

name: Foundry project
runs-on: ubuntu-latest
runs-on: [self-hosted, Linux, medium, ephemeral]
steps:
- uses: actions/checkout@v3
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
submodules: recursive

- name: Install Foundry
uses: foundry-rs/foundry-toolchain@v1
uses: foundry-rs/foundry-toolchain@8f1998e9878d786675189ef566a2e4bf24869773 # v1.2.0
with:
version: nightly

Expand Down
15 changes: 10 additions & 5 deletions .github/workflows/image-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,27 @@ env:

jobs:
build:
runs-on: ubuntu-latest
runs-on: [self-hosted, Linux, medium, ephemeral]

steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit

- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6

- name: Set up Docker Qemu
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@68827325e0b33c7199eb31dd4e31fbe9023e06e3 # v3.0.0

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@d70bba72b1f3fd22344832f00baa16ece964efeb # v3.3.0
with:
driver-opts: network=host

- name: Build image
uses: docker/build-push-action@v2
uses: docker/build-push-action@2cdde995de11925a030ce8070c3d77a52ffcf1c0 # v5.3.0
with:
cache-from: type=gha
cache-to: type=gha,mode=max
Expand Down
Loading

0 comments on commit e8b8359

Please sign in to comment.