Skip to content

perf(lint): isolate golangci-lint cache and lock per worktree - #2701

Merged
Andriy Knysh (aknysh) merged 3 commits into
mainfrom
osterman/optimize-custom-gcl-caching
Jul 9, 2026
Merged

Andriy Knysh (aknysh) merged 3 commits into
mainfrom
osterman/optimize-custom-gcl-caching

Conversation

@osterman

@osterman Erik Osterman (Cloud Posse) (osterman) commented Jul 8, 2026 •

Copy link
Copy Markdown
Member

what

  • Point TMPDIR and GOLANGCI_LINT_CACHE at repo-local dirs in scripts/run-custom-golangci-lint.sh so each worktree gets its own golangci-lint lock and cache (GOCACHE stays shared for warm typecheck data); opt out with ATMOS_LINT_SHARED_CACHE=1.
  • Skip rebuilding the custom-gcl binary in atmos lint custom-gcl when it is already up to date (staleness guard mirroring lint lintroller/lint gomodcheck).
  • Drop the redundant standalone lint lintroller pass from atmos lint changed (it already runs as a plugin inside custom-gcl).

why

  • golangci-lint's single-instance lock is a fixed machine-global path (os.TempDir()/golangci-lint.lock), not inside GOLANGCI_LINT_CACHE, so parallel worktree lints (e.g. rebase storms after a merge) serialized on one lock and a run orphaned by a tool timeout froze every other worktree; relocating the lock via a per-worktree TMPDIR lets them run in parallel and contains an orphan to its own worktree.
  • Rebuilding custom-gcl (clone + compile) on every lint changed was the single biggest per-run cost during a rebase storm when the binary is identical across worktrees.
  • Verified empirically: an isolated-TMPDIR lint ran at ~94% CPU in parallel with another holding the global lock, with its lock file under .golangci-tmp/.

references

  • Contributor tooling only (dev lint flow + pre-commit hook); no user-visible change to the Atmos binary — labeled no-release.

Parallel worktree lints (rebase storms) serialized on golangci-lint's
machine-global single-instance lock at $TMPDIR/golangci-lint.lock, and a run
orphaned by a tool timeout froze every other worktree. Point TMPDIR and
GOLANGCI_LINT_CACHE at repo-local dirs so each worktree gets its own lock and
cache (GOCACHE stays shared for warm typecheck data), skip rebuilding custom-gcl
when it is up to date, and drop the redundant standalone lintroller pass from
`lint changed` (it already runs via the custom-gcl plugin).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@atmos-pro

atmos-pro Bot commented Jul 8, 2026 •

Copy link
Copy Markdown
Contributor

Tip

Atmos Pro  

No affected stacks workflow was detected for this pull request.
If this is expected, no action is needed.
Learn More. Ask AI.

@osterman Erik Osterman (Cloud Posse) (osterman) added the no-release Do not create a new release (wait for additional code changes) label Jul 8, 2026
@github-actions github-actions Bot added the size/s Small size PR label Jul 8, 2026
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None

@coderabbitai

coderabbitai Bot commented Jul 8, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 4d5571bf-d351-4ec9-b663-4092385269de

📥 Commits

Reviewing files that changed from the base of the PR and between b271f89 and 3b6aaba.

📒 Files selected for processing (1)
  • .gitignore
✅ Files skipped from review due to trivial changes (1)
  • .gitignore

📝 Walkthrough

Walkthrough

The PR removes a redundant lint step, makes custom-gcl rebuild only when inputs are newer than the binary, and updates golangci-lint execution to use worktree-local cache and temp directories unless shared caching is enabled.

Changes

Lint pipeline updates

Layer / File(s) Summary
Lint step ordering and conditional custom-gcl rebuild
.atmos.d/lint.yaml
Removes the separate lint lintroller step after build deps, and changes custom-gcl to rebuild only when the binary is missing or stale relative to .custom-gcl.yml or tools/lintroller Go sources.
Per-worktree cache and lock isolation
scripts/run-custom-golangci-lint.sh, .gitignore
Sets GOLANGCI_LINT_CACHE and TMPDIR to worktree-local directories unless ATMOS_LINT_SHARED_CACHE=1, creates the temp directory, and adds .golangci-tmp/ ignore entries with updated cache comments.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant lint_changed as lint changed
  participant run_custom_golangci_lint as run-custom-golangci-lint.sh
  participant filesystem as worktree filesystem

  lint_changed->>run_custom_golangci_lint: invoke lint
  run_custom_golangci_lint->>run_custom_golangci_lint: check ATMOS_LINT_SHARED_CACHE
  run_custom_golangci_lint->>filesystem: set GOLANGCI_LINT_CACHE to .golangci-cache
  run_custom_golangci_lint->>filesystem: set TMPDIR to .golangci-tmp
  run_custom_golangci_lint->>filesystem: create .golangci-tmp directory
  run_custom_golangci_lint->>lint_changed: run golangci-lint with isolated cache and lock paths
Loading

Possibly related PRs

Suggested reviewers: aknysh

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: isolating golangci-lint cache and lock behavior per worktree for lint performance.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch osterman/optimize-custom-gcl-caching

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 8, 2026
@codecov

codecov Bot commented Jul 8, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.18%. Comparing base (b248714) to head (2cec399).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2701   +/-   ##
=======================================
  Coverage   81.18%   81.18%           
=======================================
  Files        1630     1630           
  Lines      154206   154206           
=======================================
+ Hits       125186   125193    +7     
+ Misses      22101    22093    -8     
- Partials     6919     6920    +1     
Flag Coverage Δ
unittests 81.18% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 8 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mergify

mergify Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

💥 This pull request now has conflicts. Could you fix it Erik Osterman (Cloud Posse) (@osterman)? 🙏

@mergify mergify Bot added the conflict This PR has conflicts label Jul 9, 2026
…tom-gcl-caching

# Conflicts:
#	scripts/run-custom-golangci-lint.sh
@mergify mergify Bot removed the conflict This PR has conflicts label Jul 9, 2026
@aknysh
Andriy Knysh (aknysh) merged commit 4ee0f7f into main Jul 9, 2026
85 of 88 checks passed
@atmos-pro

atmos-pro Bot commented Jul 9, 2026 •

Copy link
Copy Markdown
Contributor

Tip

Atmos Pro  

No affected stacks workflow was detected for this pull request.
If this is expected, no action is needed.
Learn More. Ask AI.

@aknysh
Andriy Knysh (aknysh) deleted the osterman/optimize-custom-gcl-caching branch July 9, 2026 12:46
@github-actions

Copy link
Copy Markdown

These changes were released in v1.223.0-rc.7.

Tobias Wolf (Wolfsrudel) pushed a commit to Wolfsrudel/iac-hc-terraform-cloudposse-atmos that referenced this pull request Aug 25, 2026
…#2988)

* perf(lint): move golangci-lint cache/tmp out of worktrees to user cache dir

In-worktree .golangci-cache/.golangci-tmp (156MB / ~37k files churning per
lint run, from cloudposse#2701) sat under filesystem watchers (git fsmonitor,
Conductor's watchexec), saturating macOS FSEvents at multi-worktree scale
and stalling every index-refreshing git command for seconds to minutes.
Relocate the default cache/tmp to <user-cache>/atmos-lint/<worktree-hash>/,
preserving cloudposse#2701's per-worktree lock isolation. ATMOS_LINT_SHARED_CACHE=1
and explicit GOLANGCI_LINT_CACHE overrides are unchanged.

See docs/fixes/2026-08-24-fsmonitor-worktree-saturation.md for the full
measured diagnosis.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(fixes): add PR reference and end-to-end timing to fsmonitor fix record

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-release Do not create a new release (wait for additional code changes) size/s Small size PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants