GitHub Stats Pipeline is DOWN

[healthchecks-io]

The check GitHub Stats Pipeline is DOWN (received a failure signal).

Project: Stats Page
Tags: stats github
Period: 1 day
Total Pings: 988
Last Ping: Exit status 1, 2 seconds ago
Last Ping Body:

Downloading cpython-3.10.21-linux-x86_64-gnu (download) (28.1MiB)
 Downloaded cpython-3.10.21-linux-x86_64-gnu (download)
Using CPython 3.10.21
Creating virtual environment at: .venv
   Building nf-core-stats @ file:///home/runner/work/stats/stats/pipeline
Downloading setuptools (1.0MiB)
Downloading pip (1.7MiB)
Downloading pygments (1.2MiB)
Downloading botocore (14.6MiB)
Downloading dlt (1.1MiB)
Downloading duckdb (19.5MiB)
Downloading pyarrow (45.3MiB)
 Downloaded pygments
 Downloaded duckdb
      Built nf-core-stats @ file:///home/runner/work/stats/stats/pipeline
 Downloaded setuptools
 Downloaded pip
 Downloaded dlt
 Downloaded pyarrow
 Downloaded botocore
Installed 63 packages in 126ms
2026-08-19 18:31:00,181 - INFO - Starting GitHub data pipeline...
2026-08-19 18:31:00,194 - INFO - Checking initial rate limit status...
2026-08-19 18:31:00,284 - INFO - Rate limit: 4970/5000 remaining (resets at 2026-08-19 18:56:00+00:00)
2026-08-19 18:31:00,322 - ERROR - Rate limited (403). resour
[truncated]

[edmundmiller]

Diagnosis

Run 32287740582 shows the github pipeline job hitting a rate-limited 403 on its very first real request (GET /orgs/nf-core/repos), just 38ms after a /rate_limit preflight read a healthy 4970/5000 remaining. The reset time on the 403 (18:35:12) didn't match the preflight's reported hourly reset (18:56:00) — a telltale sign of GitHub's secondary rate limit, not primary quota exhaustion.

Looking at the job timeline for that run, the citations matrix leg ran concurrently with github, both starting around 18:30:54. citations authenticates with the same GH_TOKEN_STATS_PAGE token and fires ~200 requests (2 per nf-core pipeline, for nextflow.config lookups) in about 14 seconds. That concurrent burst on the same token is what tripped the secondary limit, aborting the github leg — even though the primary quota had thousands of requests to spare.

This is a distinct issue from the primary-quota/401 problems fixed in #145, #151, and #152 — those addressed request volume and error handling; this one is about two matrix legs sharing one token running at the same time.

Fix

Pushed to claude/cool-cray-idolaq (commit 3e30918): added max-parallel: 1 to the run_pipelines matrix strategy so github, slack, citations, and newsletter never race each other against the same GitHub token. Also documented the constraint in pipeline/AGENTS.md so future GitHub-API-calling pipelines stay in the serialized matrix.

Compare: main...claude/cool-cray-idolaq

---

Generated by Claude Code