Skip to content

Commit b8275f4

Browse files
committed
ci: cache a port's submodules in an entry of their own
Every board job of a port cloned the port's submodules over the network: stm's 29 CMSIS and HAL repositories take 59 seconds a job, raspberrypi's sdk, lwip, cyw43 and PicoDVI 17 seconds, 159 times per full run. The common submodule cache holds extmod/ulab, lib/ and tools/ and stays as it is; board jobs add a second entry with ports/<port>, keyed by the port and the state of its submodules. espressif and zephyr-cp keep their own arrangements.
1 parent a156588 commit b8275f4

2 files changed

Lines changed: 28 additions & 1 deletion

File tree

‎.github/actions/deps/submodules/action.yml‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,12 @@ inputs:
1212
default: '["extmod/ulab", "lib/", "tools/"]'
1313
type: string
1414

15+
port:
16+
description: 'Also cache the submodules of this port, in their own entry'
17+
required: false
18+
default: ''
19+
type: string
20+
1521
action:
1622
description: 'The cache action to use'
1723
required: false
@@ -62,8 +68,27 @@ runs:
6268
key: submodules-common-${{ hashFiles('submodule_status') }}
6369
enableCrossOsArchive: true
6470

71+
- name: Port submodule status
72+
id: port-submodule-status
73+
# The ports whose board jobs clone the most over the network: 29 CMSIS and HAL
74+
# repositories on stm, the sdk and lwip on raspberrypi and mimxrt10xx. espressif
75+
# and zephyr-cp fetch theirs in their own way; the rest have little to gain.
76+
if: ${{ contains(fromJSON('["stm", "raspberrypi", "mimxrt10xx"]'), inputs.port) }}
77+
run: |
78+
git submodule status ports/${{ inputs.port }} > port_submodule_status
79+
echo $(cut -d ' ' -f 2 port_submodule_status) | echo "submodules=[\"$(sed "s/ /\", \"/g")\"]" >> $GITHUB_OUTPUT
80+
shell: bash
81+
82+
- name: Cache port submodules
83+
if: ${{ steps.port-submodule-status.outcome == 'success' }}
84+
uses: actions/cache@v5
85+
with:
86+
path: ".git/modules/ports/${{ inputs.port }}\n${{ join(fromJSON(steps.port-submodule-status.outputs.submodules), '\n') }}"
87+
key: submodules-${{ inputs.port }}-${{ hashFiles('port_submodule_status') }}
88+
enableCrossOsArchive: true
89+
6590
- name: Remove submodule status
66-
run: rm submodule_status
91+
run: rm -f submodule_status port_submodule_status
6792
shell: bash
6893

6994
- name: CircuitPython dependencies

‎.github/workflows/build-boards.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,8 @@ jobs:
5151
- name: Set up submodules
5252
id: set-up-submodules
5353
uses: ./.github/actions/deps/submodules
54+
with:
55+
port: ${{ inputs.port }}
5456

5557
- name: Set up external
5658
uses: ./.github/actions/deps/external

0 commit comments

Comments
 (0)