ci(github): pin all actions to commit SHAs to comply with ASF allow-list policy - #2403
ci(github): pin all actions to commit SHAs to comply with ASF allow-list policy#2403acelyc111 wants to merge 7 commits into
Conversation
…ist policy Backport of apache#2398 to v2.5, adapted to v2.5's workflow structure (which predates master's apache#1791 composite-action refactor). According to the ASF GitHub Actions Policy, third-party actions are blocked by default in apache/* repositories unless they appear in the ASF organization-wide allow-list. Only actions in the apache/* and actions/* namespaces are auto-allowed; everything else must be pinned to a commit SHA listed in apache/infrastructure-actions. On v2.5, action refs like 'dorny/paths-filter@v2', 'docker/build-push-action@v6', 'hadolint/hadolint-action@v3.1.0', etc. are not on the allow-list and cause workflow runs to end in 'startup_failure' with no jobs (no logs, no PR check). For example, the Cpp CI run for PR apache#2394 failed to start for exactly this reason (see https://github.com/apache/incubator-pegasus/actions/runs/24079456724). Pin every action ref in v2.5's workflows to a commit SHA from the current ASF allow-list. SHAs match those used on master where applicable; where master's SHA has since been expired/replaced in the ASF allow-list, pick the latest non-expired SHA from the same major version line. Two non-mechanical adjustments mirror master: - Drop the 'codecov/codecov-action@v2' step in lint_and_test_go-client (master removed it via apache#1790 in 2023; the moving '@v2' tag is also a supply-chain risk). - Replace 'gaurav-nelson/github-action-markdown-link-check@1.0.13' with 'tcort/github-action-markdown-link-check@<sha>' in standardization_lint (master switched via apache#2329 because gaurav-nelson is not in the ASF allow-list). ASF policy reference: - https://infra.apache.org/github-actions-policy.html - https://github.com/apache/infrastructure-actions - https://issues.apache.org/jira/browse/INFRA-27084
The apache/pegasus:clang-format-3.9 container is built on an older base
image whose GLIBC is < 2.28. Node 20 binaries shipped with
actions/checkout v4+ cannot start in this environment, causing the
checkout step to abort with:
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6:
version `GLIBC_2.28' not found
Pin actions/checkout to @V3 (Node 16) for this single job. actions/* is
auto-allowed by the ASF allow-list regardless of ref, so this does not
reintroduce the policy block that this PR fixes.
The previous workaround (keeping actions/checkout@v3 inside the apache/pegasus:clang-format-3.9 container) no longer works: GitHub now executes actions/checkout@v3 on Node 20, which fails to start inside the container because the image ships GLIBC < 2.28 and Node 20 requires GLIBC 2.28+: /__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node) Switch the cpp_clang_format_linter job to run directly on ubuntu-22.04 (no container), pin actions/checkout to v4.3.1 (matching the rest of the workflow), and extract clang-format-3.9 from the same image as a plain binary. This keeps clang-format 3.9 as the formatter of record for v2.5 (no codebase-wide reformatting required) while unblocking the rest of the Cpp CI matrix that depends on this job.
…ker cp -L) In the apache/pegasus:clang-format-3.9 image, /usr/bin/clang-format-3.9 is a relative symlink to ../lib/llvm-3.9/bin/clang-format. A plain docker cp copies the symlink verbatim, which lands on the host as a dangling link: invalid symlink ".../clang-format-3.9" -> "../lib/llvm-3.9/bin/clang-format" Fix by resolving the symlink to its real path inside the container with readlink -f, and by passing -L to docker cp so the binary itself is copied out.
…nstall
Extracting the clang-format-3.9 binary out of the legacy image is not
enough on its own: the binary is dynamically linked against
libLLVM-3.9.so.1, which ubuntu-22.04 no longer packages, so it fails
on the host with:
clang-format-3.9: error while loading shared libraries:
libLLVM-3.9.so.1: cannot open shared object file
Instead, install a tiny /usr/local/bin/clang-format-3.9 wrapper that
runs the real binary inside the apache/pegasus:clang-format-3.9 image
with the workspace bind-mounted at the same path, so paths produced
by run-clang-format.py resolve identically inside and outside the
container. This keeps clang-format 3.9 as the formatter of record for
v2.5 without requiring its libraries on the runner host.
The default shell on Linux runners is /bin/sh (dash on Ubuntu). Some test setup steps (notably zkServer.sh, invoked from run.sh test) rely on bash-specific behavior; under sh they fail at startup with no visible error, producing the symptom 'Starting zookeeper ... FAILED TO START' followed by exit code 1 from every Test ASAN matrix job. This previously went unnoticed on v2.5 because workflows were blocked by the ASF allow-list at startup before any job ran. Now that the allow-list block is lifted (by the previous commits in this PR), test jobs reach the run.sh stage and surface this latent issue. master already declares 'defaults.run.shell: bash' in this same workflow; v2.5 simply never picked it up. Add the same declaration so v2.5 PRs can run the C++ test matrix to completion.
|
Pushed What I observed on the previous attemptAll 31 So every ASAN test job actually got past download/build artifact, started the test runner, and then died because Why this is surfacing on this PRComparing GitHub Actions step config between this PR and the last historically-successful Cpp CI run on a v2.5-base PR (
What
|
The previous commit (defaults.run.shell: bash) did not actually fix the 'Starting zookeeper ... FAILED TO START' failure; ZK still exits ~1 second after launch. The shell change was confirmed in logs (steps now run under bash --noprofile --norc -e -o pipefail), but ZK fails the same way, so the root cause is elsewhere — likely inside the JVM, which writes its real stderr to zookeeper.out (not surfaced by GitHub Actions). Add an 'if: failure()' diagnostics step to both Test ASAN and Test Release jobs that dumps: - java -version, JAVA_HOME, ulimit - .zk_install/zookeeper-bin/ tree - All candidate locations of zookeeper.out - zoo.cfg This is a TEMPORARY DIAGNOSTIC commit. Once the actual ZK failure cause is identified (and presumably fixed in a follow-up), this commit should be reverted before this PR is merged.
Update on the Test ASAN/Release failuresAfter looking deeper, the Key finding: this is a pre-existing v2.5 problem, not a regression from this PRI'd been mentally comparing this PR against the most recent successful Querying the
v2.5's only run of that workflow was a So Concrete evidence the image is the problem and not anything in this PR
What I just pushed (
|
Summary
Backport of #2398 to v2.5, adapted to v2.5's workflow structure (which predates master's #1791 composite-action refactor).
Why this is needed
The ASF GitHub Actions Policy (infra.apache.org/github-actions-policy.html, INFRA-27084) blocks third-party actions in
apache/*repositories unless they appear in the ASF organization-wide allow-list at apache/infrastructure-actions. Only actions inapache/*andactions/*namespaces are auto-allowed; everything else must be pinned to a commit SHA explicitly listed in that allow-list.On v2.5, action refs like
dorny/paths-filter@v2,docker/build-push-action@v6,hadolint/hadolint-action@v3.1.0,gaurav-nelson/github-action-markdown-link-check@1.0.13, etc. are not on the allow-list, so workflow runs end inconclusion: startup_failurewith an empty jobs array — meaning no logs, no PR check entry, the workflow appears "silent" but is actually being blocked. This is exactly what happens to existing v2.5 PRs like #2394, where theCpp CIrun 24079456724 failed to start with the message:What this PR does
Pin every action ref in v2.5's workflows to a commit SHA approved by the current ASF allow-list. SHAs match those used on master (#2398) where the ASF allow-list still accepts them; where master's SHA has since been expired/replaced in the allow-list, the latest non-expired SHA from the same major version line is used (notably
docker/setup-qemu-action@v3.7.0andgolangci/golangci-lint-action@v9.2.1).Two non-mechanical adjustments mirror master:
codecov/codecov-action@v2step inlint_and_test_go-client.yml— master removed it in chore(go-client-ci): build the server instead of downloading it from external site #1790 (Dec 2023); v2.5 was never updated. The moving@v2tag is also a supply-chain risk.gaurav-nelson/github-action-markdown-link-check@1.0.13→tcort/github-action-markdown-link-check@<sha>instandardization_lint.yaml— master switched via ci(github): update markdown lint check #2329 because gaurav-nelson is not in the ASF allow-list. Both actions take zero parameters here, so the swap is API-compatible.apache/skywalking-eyes@mainis intentionally left untouched (master also keeps@main; theapache/*namespace is auto-allowed regardless of ref).One job uses
actions/checkout@v3instead of@v4.3.1cpp_clang_format_linterruns inside theapache/pegasus:clang-format-3.9container. That container's base image ships GLIBC < 2.28, which cannot run the Node 20 binaries shipped withactions/checkout@v4+. The check fails with:The clean solution on master was #2063 (Jul 2024): bump
clang-formatfrom 3.9 to 14, switch the job to a plainubuntu-22.04runner, and delete theapache/pegasus:clang-format-3.9image. That refactor reformats the entire C++ codebase and is intentionally out of scope for this PR — it deserves a separate v2.5 backport. For now this PR keepsactions/checkout@v3in just this one job;actions/*is auto-allowed by the ASF policy regardless of ref, so this does not reintroduce the policy block.Verification
yaml.safe_load)uses:ref in this branch was checked against ASFapproved_patterns.ymlfromapache/infrastructure-actions— all pass# vendor/name@vX.Y.Zcomment lines preceding each pinned SHA (matching master's ci(github): pin all actions to exact commit SHAs instead of tags or branch references #2398 style)Cpp CIruns to a real result on this PR (no morestartup_failure) — see CI checks belowCI status of this PR
After pushing this branch, several workflows that were previously silently blocked (
Cpp CI,Standardization Lint) now actually run, which directly proves the ASF policy block has been lifted. Three CI failures remain; only one (Cpp Lint) is something this PR introduced and addresses, the other two are pre-existing or unrelated:Cpp CI / Lintactions/checkout@v4.3.1(Node 20) which is incompatible with theclang-format-3.9container's GLIBC < 2.28. Fixed by keeping@v3for that one job (see "Why one job uses @V3" above).Standardization Lint / Lint PR title[backport]whichamannn/action-semantic-pull-requestrejects as not a valid Conventional Commits prefix. Fixed by editing the title.Labeler / Module LabelerResource not accessible by integration(the action lacks write permission to add labels). This is a pre-existing fork-PR limitation:pull_request_targeton Apache repos does not grant write tokens to forks for security reasons. Not introduced by this PR. The same failure would occur on any v2.5 PR from a fork. Out of scope here.What this PR does NOT do
.github/actions/build_pegasus, etc. from chore(CI): Improve github actions #1791/chore(CI): Improve github actions (2nd) #1812). v2.5's workflows remain self-contained, which is the minimum-risk change for a release branch.Related
Cpp CIstartup_failure