Skip to content

Commit 28405e3

Browse files
justin808justin808
authored
Add CI debugging tools for efficient local failure reproduction (#1973)
## Problem Debugging CI failures currently requires: 1. Waiting for CI to complete 2. Manually identifying which jobs failed 3. Figuring out which local commands replicate those jobs 4. Running entire test suites when only a few specs failed This is slow and inefficient. ## Solution Adds two powerful scripts that automate CI failure reproduction: ### 1. `bin/ci-rerun-failures` Automatically detects and re-runs only failed CI jobs: ```bash # Run failed jobs from current commit bin/ci-rerun-failures # Search recent commits for failures bin/ci-rerun-failures --previous # Run failures from specific PR bin/ci-rerun-failures 1964 ``` **Features:** - ✨ Fetches actual CI failures from GitHub using `gh` CLI - 🎯 Runs only what failed (no wasted time on passing tests) - ⏳ Waits for in-progress CI or searches previous commits - 📋 Shows exactly what will run before executing - 🚀 Maps CI job names to local rake commands automatically ### 2. `bin/ci-run-failed-specs` Runs only specific failing RSpec examples: ```bash # Copy failure output from CI, then: pbpaste | bin/ci-run-failed-specs # Or pass spec paths directly: bin/ci-run-failed-specs './spec/system/integration_spec.rb[1:1:1:1]' ``` **Features:** - 🎯 Runs only failing examples (not entire test suite) - 📋 Parses RSpec output to extract spec paths - 🔄 Deduplicates specs - 📁 Auto-detects spec/dummy directory when needed ## Benefits - **Faster iteration**: Run exactly what failed, skip what passed - **No more guessing**: Scripts map CI jobs to local commands - **No waiting**: Replicate failures immediately without CI feedback loop - **Surgical precision**: Run individual failing specs, not entire suites ## Documentation Added comprehensive usage guide to CLAUDE.md with examples. --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> <!-- Reviewable:start --> - - - This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/shakacode/react_on_rails/1973) <!-- Reviewable:end --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added comprehensive guides for replicating CI failures locally, including configuration switching workflows and troubleshooting guidance. * Documented local system test setup and SSL-related considerations. * **Chores** * Added developer scripts for re-running failed CI jobs, switching between configurations, and executing specific test examples locally. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent d1a8a1a commit 28405e3

File tree

6 files changed

+1195
-0
lines changed

6 files changed

+1195
-0
lines changed

‎CLAUDE.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,75 @@ Pre-commit hooks automatically run:
4848
- **⚠️ MANDATORY BEFORE GIT PUSH**: `bundle exec rubocop` and fix ALL violations + ensure trailing newlines
4949
- Never run `npm` commands, only equivalent Yarn Classic ones
5050

51+
### Replicating CI Failures Locally
52+
53+
**CRITICAL: NEVER wait for CI to verify fixes. Always replicate failures locally first.**
54+
55+
#### Switch Between CI Configurations
56+
57+
The project tests against two configurations:
58+
- **Latest**: Ruby 3.4, Node 22, Shakapacker 9.3.0, React 19 (runs on all PRs)
59+
- **Minimum**: Ruby 3.2, Node 20, Shakapacker 8.2.0, React 18 (runs only on master)
60+
61+
```bash
62+
# Check your current configuration
63+
bin/ci-switch-config status
64+
65+
# Switch to minimum dependencies (for debugging minimum CI failures)
66+
bin/ci-switch-config minimum
67+
68+
# Switch back to latest dependencies
69+
bin/ci-switch-config latest
70+
```
71+
72+
**See `SWITCHING_CI_CONFIGS.md` for detailed usage and troubleshooting.**
73+
74+
**See `spec/dummy/TESTING_LOCALLY.md` for local testing tips and known issues.**
75+
76+
#### Re-run Failed CI Jobs
77+
78+
```bash
79+
# Automatically detects and re-runs only the failed CI jobs
80+
bin/ci-rerun-failures
81+
82+
# Search recent commits for failures (when current commit is clean/in-progress)
83+
bin/ci-rerun-failures --previous
84+
85+
# Or for a specific PR number
86+
bin/ci-rerun-failures 1964
87+
```
88+
89+
This script:
90+
- ✨ **Fetches actual CI failures** from GitHub using `gh` CLI
91+
- 🎯 **Runs only what failed** - no wasted time on passing tests
92+
- ⏳ **Waits for in-progress CI** - offers to poll until completion
93+
- 🔍 **Searches previous commits** - finds failures before your latest push
94+
- 📋 **Shows you exactly what will run** before executing
95+
- 🚀 **Maps CI jobs to local commands** automatically
96+
97+
#### Run Only Failed Examples
98+
99+
When RSpec tests fail, run just those specific examples:
100+
101+
```bash
102+
# Copy failure output from GitHub Actions, then:
103+
pbpaste | bin/ci-run-failed-specs # macOS
104+
# xclip -o | bin/ci-run-failed-specs # Linux (requires: apt install xclip)
105+
# wl-paste | bin/ci-run-failed-specs # Wayland (requires: apt install wl-clipboard)
106+
107+
# Or pass spec paths directly:
108+
bin/ci-run-failed-specs './spec/system/integration_spec.rb[1:1:1:1]'
109+
110+
# Or from a file:
111+
bin/ci-run-failed-specs < failures.txt
112+
```
113+
114+
This script:
115+
- 🎯 **Runs only failing examples** - not the entire test suite
116+
- 📋 **Parses RSpec output** - extracts spec paths automatically
117+
- 🔄 **Deduplicates** - removes duplicate specs
118+
- 📁 **Auto-detects directory** - runs from spec/dummy when needed
119+
51120
## Changelog
52121

53122
- **Update CHANGELOG.md for user-visible changes only** (features, bug fixes, breaking changes, deprecations, performance improvements)

‎SWITCHING_CI_CONFIGS.md‎

Lines changed: 299 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,299 @@
1+
# Switching Between CI Configurations Locally
2+
3+
This guide explains how to switch between different CI test configurations locally to replicate CI failures.
4+
5+
## Quick Start
6+
7+
```bash
8+
# Check your current configuration
9+
bin/ci-switch-config status
10+
11+
# Switch to minimum dependencies (Ruby 3.2, Node 20)
12+
bin/ci-switch-config minimum
13+
14+
# Switch back to latest dependencies (Ruby 3.4, Node 22)
15+
bin/ci-switch-config latest
16+
```
17+
18+
## CI Configurations
19+
20+
The project runs tests against two configurations:
21+
22+
### Latest (Default Development)
23+
24+
- **Ruby**: 3.4
25+
- **Node**: 22
26+
- **Shakapacker**: 9.3.0
27+
- **React**: 19.0.0
28+
- **Dependencies**: Latest versions with `--frozen-lockfile`
29+
- **When it runs**: Always on PRs and master
30+
31+
### Minimum (Compatibility Testing)
32+
33+
- **Ruby**: 3.2
34+
- **Node**: 20
35+
- **Shakapacker**: 8.2.0
36+
- **React**: 18.0.0
37+
- **Dependencies**: Minimum supported versions
38+
- **When it runs**: Only on master branch
39+
40+
## When to Switch Configurations
41+
42+
**Switch to minimum when:**
43+
44+
- CI fails on `dummy-app-integration-tests (3.2, 20, minimum)` but passes on latest
45+
- You're debugging compatibility with older dependencies
46+
- You want to verify minimum version support before releasing
47+
48+
**Switch to latest when:**
49+
50+
- You're done testing minimum configuration
51+
- You want to return to normal development
52+
- CI failures are on latest configuration
53+
54+
## Prerequisites
55+
56+
You must have a version manager like [mise](https://mise.jdx.dev/) (recommended) or [asdf](https://asdf-vm.com/) installed to manage Ruby and Node versions.
57+
58+
```bash
59+
# Install mise (recommended, modern alternative to asdf)
60+
brew install mise
61+
echo 'eval "$(mise activate zsh)"' >> ~/.zshrc
62+
source ~/.zshrc
63+
64+
# OR install asdf (legacy option)
65+
brew install asdf
66+
echo -e "\n. $(brew --prefix asdf)/libexec/asdf.sh" >> ~/.zshrc
67+
source ~/.zshrc
68+
69+
# Install plugins (only needed for asdf, mise reads from mise.toml)
70+
asdf plugin add ruby
71+
asdf plugin add nodejs
72+
```
73+
74+
## Detailed Usage
75+
76+
### 1. Check Current Configuration
77+
78+
```bash
79+
bin/ci-switch-config status
80+
```
81+
82+
This shows:
83+
84+
- Current Ruby and Node versions
85+
- Dependency versions (Shakapacker, React)
86+
- Which configuration you're currently on
87+
88+
### 2. Switch to Minimum Configuration
89+
90+
```bash
91+
bin/ci-switch-config minimum
92+
```
93+
94+
This will:
95+
96+
1. Create `.tool-versions` with Ruby 3.2.8 and Node 20.18.1
97+
2. Run `script/convert` to downgrade dependencies:
98+
- Shakapacker 9.3.0 → 8.2.0
99+
- React 19.0.0 → 18.0.0
100+
- Remove ESLint and other packages incompatible with Node 20
101+
3. Clean `node_modules` and `yarn.lock`
102+
4. Reinstall dependencies without `--frozen-lockfile`
103+
5. Clean and reinstall spec/dummy dependencies
104+
105+
**After switching, run:**
106+
107+
```bash
108+
# Reload your shell to pick up new Ruby/Node versions
109+
cd <project-root>
110+
mise current # or: asdf current
111+
112+
# Build and test
113+
rake node_package
114+
cd spec/dummy
115+
bin/shakapacker-precompile-hook
116+
RAILS_ENV=test bin/shakapacker
117+
cd ../..
118+
bundle exec rake run_rspec:all_dummy
119+
```
120+
121+
### 3. Switch Back to Latest Configuration
122+
123+
```bash
124+
bin/ci-switch-config latest
125+
```
126+
127+
This will:
128+
129+
1. Create `.tool-versions` with Ruby 3.4.3 and Node 22.12.0
130+
2. Restore files from git (reverting changes made by `script/convert`)
131+
3. Clean `node_modules` and `yarn.lock`
132+
4. Reinstall dependencies with `--frozen-lockfile`
133+
5. Clean and reinstall spec/dummy dependencies
134+
135+
**After switching, run:**
136+
137+
```bash
138+
# Reload your shell to pick up new Ruby/Node versions
139+
cd <project-root>
140+
mise current # or: asdf current
141+
142+
# Build and test
143+
rake node_package
144+
cd spec/dummy
145+
bin/shakapacker-precompile-hook
146+
RAILS_ENV=test bin/shakapacker
147+
cd ../..
148+
bundle exec rake run_rspec:all_dummy
149+
```
150+
151+
## What Gets Modified
152+
153+
When switching to **minimum**, these files are modified:
154+
155+
- `.tool-versions` - Ruby/Node versions
156+
- `Gemfile.development_dependencies` - Shakapacker gem version
157+
- `package.json` - React versions, dev dependencies removed
158+
- `spec/dummy/package.json` - React and Shakapacker versions
159+
- `packages/react-on-rails-pro/package.json` - Test scripts modified
160+
- `node_modules/`, `yarn.lock` - Cleaned and regenerated
161+
- `spec/dummy/node_modules/`, `spec/dummy/yarn.lock` - Cleaned and regenerated
162+
163+
When switching to **latest**, these files are restored from git.
164+
165+
## Common Workflows
166+
167+
### Debugging a Minimum Config CI Failure
168+
169+
```bash
170+
# 1. Check current config
171+
bin/ci-switch-config status
172+
173+
# 2. Switch to minimum
174+
bin/ci-switch-config minimum
175+
176+
# 3. Reload shell
177+
cd <project-root>
178+
179+
# 4. Verify versions changed
180+
ruby --version # Should show 3.2.x
181+
node --version # Should show v20.x
182+
183+
# 5. Build and test
184+
rake node_package
185+
cd spec/dummy
186+
bin/shakapacker-precompile-hook
187+
RAILS_ENV=test bin/shakapacker
188+
cd ../..
189+
190+
# 6. Run the failing tests
191+
bundle exec rake run_rspec:all_dummy
192+
193+
# 7. Fix the issue
194+
195+
# 8. Switch back when done
196+
bin/ci-switch-config latest
197+
```
198+
199+
### Quick Test in Both Configurations
200+
201+
```bash
202+
# Test in latest (current default)
203+
bin/ci-switch-config status
204+
bundle exec rake run_rspec:all_dummy
205+
206+
# Switch and test in minimum
207+
bin/ci-switch-config minimum
208+
rake node_package
209+
cd spec/dummy && bin/shakapacker-precompile-hook && RAILS_ENV=test bin/shakapacker && cd ../..
210+
bundle exec rake run_rspec:all_dummy
211+
212+
# Switch back
213+
bin/ci-switch-config latest
214+
```
215+
216+
## Troubleshooting
217+
218+
### "No version is set for ruby" or version didn't change
219+
220+
After switching, you need to reload your shell:
221+
222+
```bash
223+
cd <project-root>
224+
# The cd command will trigger mise/asdf to load the new versions
225+
ruby --version # Verify it changed
226+
```
227+
228+
### Ruby/Node version didn't change
229+
230+
If your version manager doesn't automatically switch:
231+
232+
**For mise:**
233+
234+
```bash
235+
mise install # Install missing versions from mise.toml or .tool-versions
236+
```
237+
238+
**For asdf:**
239+
240+
```bash
241+
asdf install # Install missing versions from .tool-versions
242+
asdf reshim ruby
243+
asdf reshim nodejs
244+
```
245+
246+
### Yarn install fails
247+
248+
If you get package resolution errors:
249+
250+
```bash
251+
# Clean everything and try again
252+
rm -rf node_modules yarn.lock spec/dummy/node_modules spec/dummy/yarn.lock
253+
yarn install
254+
cd spec/dummy && yarn install
255+
```
256+
257+
### Git complains about modified files
258+
259+
The script will warn you if you have uncommitted changes. You can:
260+
261+
- Commit or stash your changes first, OR
262+
- Proceed (script will ask for confirmation)
263+
264+
### Switching back doesn't restore everything
265+
266+
If `git restore` doesn't work:
267+
268+
```bash
269+
# Manually restore from git
270+
git restore Gemfile.development_dependencies package.json spec/dummy/package.json packages/react-on-rails-pro/package.json
271+
272+
# Then run latest again
273+
bin/ci-switch-config latest
274+
```
275+
276+
## Integration with CI Debugging Tools
277+
278+
This script works well with the other CI debugging tools:
279+
280+
```bash
281+
# 1. Check what failed in CI
282+
bin/ci-rerun-failures
283+
284+
# 2. If it's a minimum config failure, switch
285+
bin/ci-switch-config minimum
286+
287+
# 3. Run the specific failing tests
288+
pbpaste | bin/ci-run-failed-specs
289+
290+
# 4. Switch back when done
291+
bin/ci-switch-config latest
292+
```
293+
294+
## See Also
295+
296+
- `CLAUDE.md` - Main development guide with CI debugging info
297+
- `bin/ci-rerun-failures` - Re-run only failed CI jobs locally
298+
- `bin/ci-run-failed-specs` - Run specific failing RSpec examples
299+
- `bin/ci-local` - Smart test detection based on changes

0 commit comments

Comments
 (0)