chore: rename compliance workflow to "Org Standards Compliance Audit" - #264
chore: rename compliance workflow to "Org Standards Compliance Audit"#264don-petry wants to merge 3 commits into
Conversation
Passing a growing JSON array via `jq --argjson` exceeds the Linux kernel's ARG_MAX limit (~2-3 MB) once enough PR data accumulates. The daily run hit this at 252 PRs across 8 repos (exit code 126, "Argument list too long"). Switch both the PR and issues accumulation loops to the NDJSON pattern already used in the Behind-Base Detection section: emit one compact JSON line per item into a string variable, then slurp into an array once at the end with `jq -cs '.'`. No PR/issue data ever passes through a command-line argument. Regression test (8 repos × 35 PRs = 280 synthetic PRs): - Old pattern: Argument list too long at repo 7/8 - New pattern: 280 PRs accumulated successfully Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…< style Address Gemini review on PR #258: - collect_classify_prs: replace --argjson page accumulation (all_nodes) with NDJSON pattern; protects against repos with many PR pages, not just large org totals. Also switch echo | jq to <<< throughout the function. - Merge activity: write ORG_MERGES/PERSONAL_MERGES to DATA_DIR/merges.json via printf (bash builtin, no exec, no ARG_MAX) so MERGE_DAILY and MERGE_BY_REPO_DAY read from a file descriptor instead of --argjson args. - All remaining echo "$x" | jq and printf | jq replaced with <<< herestrings as suggested (more idiomatic, avoids echo flag interpretation edge case). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
|
Closing — branch was cut from the wrong base (included org_status.sh commits already merged via #258). Will reopen from main. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors scripts/org_status.sh to mitigate ARG_MAX limitations when processing large volumes of pull requests and issues. It replaces iterative JSON array concatenation with NDJSON accumulation and utilizes temporary files for merge data. Review feedback highlights potential
| all_nodes_ndjson+=$(jq -c '.[]' <<< "$nodes") | ||
| all_nodes_ndjson+=$'\n' |
There was a problem hiding this comment.
Appending to a shell variable in a loop (e.g., all_nodes_ndjson+=$'\n') leads to $DATA_DIR, a more scalable approach is to append the NDJSON output directly to a file (e.g., >> "$DATA_DIR/nodes.ndjson") and then have the final jq command read from that file.
| --argjson org "$ORG_MERGES" --argjson personal "$PERSONAL_MERGES" ' | ||
| # Write merges to a temp file so subsequent jq calls read from a file descriptor | ||
| # rather than shell arguments — avoids the same ARG_MAX risk at high merge volumes. | ||
| printf '{"org":%s,"personal":%s}\n' "$ORG_MERGES" "$PERSONAL_MERGES" > "$DATA_DIR/merges.json" |
There was a problem hiding this comment.
While using printf to write to a file avoids the ARG_MAX limit for external commands, passing very large strings as arguments to the printf builtin still consumes significant memory and may hit shell-specific limits. A more robust approach for handling large datasets is to redirect the output of the gh commands directly to files (e.g., gh search ... > "$DATA_DIR/org_merges.json") and then use jq --slurpfile to process them, avoiding the use of large shell variables entirely.
There was a problem hiding this comment.
Pull request overview
Updates org automation by renaming the compliance workflow’s display name, and refactors org_status.sh to avoid ARG_MAX failures by switching several JSON accumulations from “growing array in a shell variable” to NDJSON + jq -s (and using a temp JSON file for merge stats).
Changes:
- Renamed workflow display name to Org Standards Compliance Audit.
- Refactored PR/issue aggregation in
scripts/org_status.shto accumulate NDJSON and slurp withjq -sto avoidARG_MAX. - Wrote merge search results to a temp JSON file and updated downstream
jqcalls to read from that file.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
scripts/org_status.sh |
Refactors JSON aggregation strategy (NDJSON + slurp; temp file for merges) to avoid command-line size limits. |
.github/workflows/compliance-audit-and-improvement.yml |
Renames the workflow display name shown in GitHub Actions UI. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Accumulate as NDJSON then slurp — avoids passing a growing $ALL_PRS as a | ||
| # shell argument to jq, which hits ARG_MAX once the org accumulates ~200+ PRs. | ||
| ALL_PR_NDJSON="" |
| # rather than shell arguments — avoids the same ARG_MAX risk at high merge volumes. | ||
| printf '{"org":%s,"personal":%s}\n' "$ORG_MERGES" "$PERSONAL_MERGES" > "$DATA_DIR/merges.json" | ||
|
|
||
| MERGE_DAILY=$(jq --arg since "$SINCE" --arg today "$TODAY" ' |
Summary
Weekly Compliance & Health AudittoOrg Standards Compliance Audit.Test plan
🤖 Generated with Claude Code