Skip to content

Commit c2bec38

Browse files
mithromithro
authored
Merge be0deb2 into 5f8d26f 
2 parents 5f8d26f + be0deb2 commit c2bec38

File tree

10 files changed

+413
-207
lines changed

10 files changed

+413
-207
lines changed

‎.claude/settings.local.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"permissions": {
3+
"allow": [
4+
"Bash(gh run list:*)",
5+
"Bash(gh run view:*)",
6+
"Bash(wget:*)",
7+
"Bash(tar:*)",
8+
"Bash(./muffet --help)",
9+
"Bash(curl:*)"
10+
],
11+
"deny": []
12+
}
13+
}

‎.github/workflows/README.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# GitHub Actions Workflows
2+
3+
This directory contains GitHub Actions workflows for automated testing and quality assurance.
4+
5+
## link-check.yml
6+
7+
**Purpose:** Validates all links in documentation previews on ReadTheDocs.
8+
9+
**Triggers:**
10+
- Pull requests opened, synchronized, or reopened against `main` branch
11+
12+
**What it does:**
13+
1. **Waits for ReadTheDocs Preview:** Monitors the ReadTheDocs build process and waits for the PR preview to become available
14+
2. **Runs Link Checker:** Uses [muffet](https://github.com/raviqqe/muffet) to crawl all pages and validate both internal and external links
15+
3. **Reports Results:** Comments on the PR with detailed results and uploads full logs as artifacts
16+
4. **Fails PR if broken links found:** Prevents merging documentation with broken links
17+
18+
**Configuration:**
19+
- **Timeout:** 15 minutes total (10 minutes to wait for ReadTheDocs + 5 minutes for link checking)
20+
- **Request timeout:** 60 seconds per request to handle slow sites
21+
- **Rate limiting:** 5 requests/second to be respectful to external sites
22+
- **User agent:** Realistic browser user agent to avoid 403 errors
23+
- **Browser headers:** Accept, Accept-Language, Accept-Encoding headers
24+
- **TLS verification:** Skipped to handle certificate issues
25+
- **Redirects:** Automatically followed
26+
- **Excluded domains:** Sites that commonly block crawlers or have reliability issues:
27+
- Social media: LinkedIn, Twitter/X, Facebook, Discord
28+
- Academic/subscription: IEEE Xplore, Reddit
29+
- Development tools: SourceForge, Efabless platform, KLayout
30+
- Community sites: SkyWater Slack invites
31+
- Corporate sites: GlobalFoundries, All About Circuits (block crawlers)
32+
33+
**ReadTheDocs URL Pattern:**
34+
The workflow constructs preview URLs using the pattern:
35+
```
36+
https://wafer-space--{PR_NUMBER}.org.readthedocs.build/en/{PR_NUMBER}/
37+
```
38+
39+
**Artifacts:**
40+
- Full muffet output is saved as workflow artifacts for 30 days
41+
- Includes detailed information about any broken links found
42+
43+
**Benefits:**
44+
- **Prevents broken links:** Catches broken links before they reach main branch
45+
- **Validates external references:** Ensures external documentation links remain valid
46+
- **Improves user experience:** Users won't encounter 404s in documentation
47+
- **Automated quality assurance:** No manual link checking required
48+
49+
**Example Results:**
50+
- ✅ **PASSED:** All links are valid, PR can be merged
51+
- ❌ **FAILED:** Broken links found, detailed report provided in PR comment
52+
53+
**Troubleshooting:**
54+
- If ReadTheDocs preview doesn't load within 10 minutes, check ReadTheDocs build status
55+
- If external links fail intermittently, they may be temporarily unavailable or blocking crawlers
56+
- Check the full muffet output in artifacts for detailed error information

‎.github/workflows/link-check.yml‎

Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
name: Link Check on ReadTheDocs Preview
2+
3+
on:
4+
pull_request:
5+
types: [opened, synchronize, reopened]
6+
branches: [main]
7+
8+
permissions:
9+
contents: read
10+
pull-requests: write
11+
issues: write
12+
13+
jobs:
14+
wait-for-readthedocs:
15+
name: Wait for ReadTheDocs Preview and Check Links
16+
runs-on: ubuntu-latest
17+
timeout-minutes: 15
18+
19+
steps:
20+
- name: Checkout repository
21+
uses: actions/checkout@v4
22+
23+
- name: Get PR number
24+
id: pr
25+
run: echo "number=${{ github.event.number }}" >> $GITHUB_OUTPUT
26+
27+
- name: Wait for ReadTheDocs build to complete
28+
id: wait-rtd
29+
run: |
30+
PR_NUMBER="${{ steps.pr.outputs.number }}"
31+
PROJECT_SLUG="wafer-space"
32+
PREVIEW_URL="https://wafer-space--${PR_NUMBER}.org.readthedocs.build/en/${PR_NUMBER}/"
33+
34+
echo "Waiting for ReadTheDocs preview at: $PREVIEW_URL"
35+
echo "preview_url=$PREVIEW_URL" >> $GITHUB_OUTPUT
36+
37+
# Function to check if URL returns 200
38+
check_url() {
39+
local url=$1
40+
local status_code=$(curl -s -o /dev/null -w "%{http_code}" "$url" || echo "000")
41+
echo "$status_code"
42+
}
43+
44+
# Wait up to 10 minutes for the preview to be available
45+
MAX_ATTEMPTS=60
46+
SLEEP_INTERVAL=10
47+
ATTEMPT=1
48+
49+
echo "Checking ReadTheDocs preview availability..."
50+
while [ $ATTEMPT -le $MAX_ATTEMPTS ]; do
51+
STATUS_CODE=$(check_url "$PREVIEW_URL")
52+
53+
if [ "$STATUS_CODE" = "200" ]; then
54+
echo "✅ ReadTheDocs preview is available!"
55+
echo "Preview URL: $PREVIEW_URL"
56+
exit 0
57+
else
58+
echo "⏳ Attempt $ATTEMPT/$MAX_ATTEMPTS: Preview not ready (HTTP $STATUS_CODE)"
59+
if [ $ATTEMPT -eq $MAX_ATTEMPTS ]; then
60+
echo "❌ ReadTheDocs preview did not become available within timeout"
61+
echo "Final status code: $STATUS_CODE"
62+
exit 1
63+
fi
64+
sleep $SLEEP_INTERVAL
65+
ATTEMPT=$((ATTEMPT + 1))
66+
fi
67+
done
68+
69+
- name: Install muffet
70+
run: |
71+
# Install muffet link checker
72+
wget -q https://github.com/raviqqe/muffet/releases/latest/download/muffet_linux_amd64.tar.gz
73+
tar -xzf muffet_linux_amd64.tar.gz
74+
sudo mv muffet /usr/local/bin/
75+
muffet --version
76+
77+
- name: Run link checker with muffet
78+
id: link-check
79+
run: |
80+
PREVIEW_URL="${{ steps.wait-rtd.outputs.preview_url }}"
81+
82+
echo "🔍 Running link checker on: $PREVIEW_URL"
83+
84+
# Create results directory
85+
mkdir -p link-check-results
86+
87+
# Run muffet with comprehensive options
88+
set +e # Don't exit on muffet errors, we want to capture them
89+
90+
muffet \
91+
--verbose \
92+
--buffer-size=8192 \
93+
--max-connections=10 \
94+
--max-connections-per-host=2 \
95+
--rate-limit=5 \
96+
--timeout=60 \
97+
--ignore-fragments \
98+
--skip-tls-verification \
99+
--max-redirections=10 \
100+
--color=never \
101+
--header="Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" \
102+
--header="Accept-Language: en-US,en;q=0.5" \
103+
--header="Accept-Encoding: gzip, deflate" \
104+
--header="User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" \
105+
--exclude=".*linkedin\.com.*" \
106+
--exclude=".*twitter\.com.*" \
107+
--exclude=".*x\.com.*" \
108+
--exclude=".*facebook\.com.*" \
109+
--exclude=".*discord\.gg.*" \
110+
--exclude=".*ieeexplore\.ieee\.org.*" \
111+
--exclude=".*reddit\.com.*" \
112+
--exclude=".*sourceforge\.io.*" \
113+
--exclude=".*sourceforge\.net.*" \
114+
--exclude=".*platform\.efabless\.com.*" \
115+
--exclude=".*invite\.skywater\.tools.*" \
116+
--exclude=".*allaboutcircuits\.com.*" \
117+
--exclude=".*globalfoundries\.com.*" \
118+
--exclude=".*klayout\.de.*" \
119+
"$PREVIEW_URL" \
120+
> link-check-results/muffet-output.txt 2>&1
121+
122+
MUFFET_EXIT_CODE=$?
123+
124+
echo "exit_code=$MUFFET_EXIT_CODE" >> $GITHUB_OUTPUT
125+
126+
# Display results
127+
echo "📊 Link Check Results:"
128+
echo "Exit code: $MUFFET_EXIT_CODE"
129+
echo ""
130+
131+
if [ $MUFFET_EXIT_CODE -eq 0 ]; then
132+
echo "✅ All links are valid!"
133+
else
134+
echo "❌ Some links failed validation"
135+
echo ""
136+
echo "Detailed output:"
137+
cat link-check-results/muffet-output.txt
138+
fi
139+
140+
- name: Upload link check results
141+
uses: actions/upload-artifact@v4
142+
if: always()
143+
with:
144+
name: link-check-results
145+
path: link-check-results/
146+
retention-days: 30
147+
148+
- name: Comment PR with results
149+
uses: actions/github-script@v7
150+
if: always()
151+
with:
152+
script: |
153+
const fs = require('fs');
154+
const path = './link-check-results/muffet-output.txt';
155+
const exitCode = '${{ steps.link-check.outputs.exit_code }}';
156+
const previewUrl = '${{ steps.wait-rtd.outputs.preview_url }}';
157+
158+
let output = '';
159+
if (fs.existsSync(path)) {
160+
output = fs.readFileSync(path, 'utf8');
161+
}
162+
163+
const success = exitCode === '0';
164+
const status = success ? '✅ PASSED' : '❌ FAILED';
165+
const emoji = success ? '🎉' : '🔍';
166+
167+
let comment = `## ${emoji} Link Check Results - ${status}\n\n`;
168+
comment += `**ReadTheDocs Preview:** ${previewUrl}\n\n`;
169+
170+
if (success) {
171+
comment += `🎉 **All links are valid!**\n\n`;
172+
comment += `The link checker found no broken links in your documentation.\n`;
173+
} else {
174+
comment += `⚠️ **Some links failed validation**\n\n`;
175+
176+
if (output) {
177+
// Truncate output if too long for GitHub comment
178+
const maxLength = 30000;
179+
let truncatedOutput = output;
180+
if (output.length > maxLength) {
181+
truncatedOutput = output.substring(0, maxLength) + '\n\n... (output truncated, see full results in artifacts)';
182+
}
183+
184+
comment += `<details><summary>🔍 Click to view detailed results</summary>\n\n`;
185+
comment += '```\n' + truncatedOutput + '\n```\n\n';
186+
comment += `</details>\n\n`;
187+
}
188+
189+
comment += `📊 Full results are available in the [workflow artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}).\n`;
190+
}
191+
192+
comment += `\n---\n`;
193+
comment += `*Link check performed by [muffet](https://github.com/raviqqe/muffet) on commit ${{ github.sha }}*`;
194+
195+
// Find existing comment to update or create new one
196+
const { data: comments } = await github.rest.issues.listComments({
197+
owner: context.repo.owner,
198+
repo: context.repo.repo,
199+
issue_number: context.issue.number
200+
});
201+
202+
const existingComment = comments.find(comment =>
203+
comment.body.includes('Link Check Results')
204+
);
205+
206+
if (existingComment) {
207+
await github.rest.issues.updateComment({
208+
owner: context.repo.owner,
209+
repo: context.repo.repo,
210+
comment_id: existingComment.id,
211+
body: comment
212+
});
213+
} else {
214+
await github.rest.issues.createComment({
215+
owner: context.repo.owner,
216+
repo: context.repo.repo,
217+
issue_number: context.issue.number,
218+
body: comment
219+
});
220+
}
221+
222+
- name: Fail workflow if links are broken
223+
if: steps.link-check.outputs.exit_code != '0'
224+
run: |
225+
echo "❌ Link check failed with exit code ${{ steps.link-check.outputs.exit_code }}"
226+
echo "Some links in the documentation are broken."
227+
echo "Please check the detailed output above and fix the broken links."
228+
exit 1

0 commit comments

Comments
 (0)