Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,5 @@ BASIC_AUTH_PASSWORD=
# Set AI_PROVIDER in .vars to choose which provider to use (default: anthropic).
ANTHROPIC_API_KEY=
GEMINI_API_KEY=
# Required when AI_REVIEW_PROVIDER=openrouter. Get one at https://openrouter.ai/keys.
OPENROUTER_API_KEY=
46 changes: 43 additions & 3 deletions .github/workflows/claude-code-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ env:

jobs:
review:
if: github.actor != 'dependabot[bot]' && vars.CLAUDE_REVIEW == 'true'
if: github.actor != 'dependabot[bot]' && vars.AI_REVIEW == 'true'
timeout-minutes: 10
runs-on: ubuntu-latest
permissions:
Expand All @@ -21,9 +21,49 @@ jobs:
with:
fetch-depth: 1

- name: Resolve reviewer provider settings
id: reviewer
env:
AI_REVIEW_PROVIDER: ${{ vars.AI_REVIEW_PROVIDER }}
AI_REVIEW_MODEL: ${{ vars.AI_REVIEW_MODEL }}
HAS_OPENROUTER_KEY: ${{ secrets.OPENROUTER_API_KEY != '' }}
HAS_ANTHROPIC_KEY: ${{ secrets.ANTHROPIC_API_KEY != '' }}
run: |
provider="${AI_REVIEW_PROVIDER:-openrouter}"
case "$provider" in
openrouter)
if [ "$HAS_OPENROUTER_KEY" != "true" ]; then
echo "::error::AI_REVIEW_PROVIDER=openrouter but OPENROUTER_API_KEY secret is not set"
exit 1
fi
default_model="qwen/qwen3.6-plus"
base_url="https://openrouter.ai/api/v1"
;;
anthropic)
if [ "$HAS_ANTHROPIC_KEY" != "true" ]; then
echo "::error::AI_REVIEW_PROVIDER=anthropic but ANTHROPIC_API_KEY secret is not set"
exit 1
fi
default_model="claude-sonnet-4-6"
base_url=""
;;
*)
echo "::error::Unknown AI_REVIEW_PROVIDER '$provider' (expected: openrouter | anthropic)"
exit 1
;;
esac
model="${AI_REVIEW_MODEL:-$default_model}"
{
echo "provider=$provider"
echo "model=$model"
echo "base_url=$base_url"
} >> "$GITHUB_OUTPUT"

- uses: anthropics/claude-code-action@v1
env:
ANTHROPIC_BASE_URL: ${{ steps.reviewer.outputs.base_url }}
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
anthropic_api_key: ${{ steps.reviewer.outputs.provider == 'openrouter' && secrets.OPENROUTER_API_KEY || secrets.ANTHROPIC_API_KEY }}
track_progress: true
prompt: |
REPO: ${{ github.repository }}
Expand All @@ -45,4 +85,4 @@ jobs:
Only post GitHub review/comments - don't submit review text as messages.

claude_args: |
--model claude-sonnet-4-6 --allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr review:*),Bash(gh pr diff:*),Bash(gh pr view:*)"
--model ${{ steps.reviewer.outputs.model }} --allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr review:*),Bash(gh pr diff:*),Bash(gh pr view:*)"
9 changes: 8 additions & 1 deletion .vars.example
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# In real GitHub Actions these are set under Settings → Variables → Actions.
# Copy to .vars and set each variable to "true" to enable the corresponding workflow or feature.
# When absent or any other value the workflow job or feature is skipped.
CLAUDE_REVIEW=
AI_REVIEW=
PLAYWRIGHT_TYPESCRIPT=
BRUNO=
QUALITY_METRICS=
Expand All @@ -16,6 +16,13 @@ AI_MODEL_FAST=
# Override the strong-tier model used for selector-fix proposals. Leave empty to use the provider default.
# Defaults: anthropic -> claude-sonnet-4-6, gemini -> gemini-3.1-flash-lite-preview.
AI_MODEL_STRONG=
# Provider for the PR code review action (claude-code-review.yml).
# Accepted values: openrouter (default), anthropic.
# openrouter requires OPENROUTER_API_KEY secret; anthropic requires ANTHROPIC_API_KEY secret.
AI_REVIEW_PROVIDER=
# Override the model used by the PR reviewer. Leave empty to use the provider default.
# Defaults: openrouter -> qwen/qwen3.6-plus, anthropic -> claude-sonnet-4-6.
AI_REVIEW_MODEL=
# Self-healing selector fix workflow. Set to "true" to enable automatic draft PRs / PR comments
# when Playwright tests fail due to locator/selector errors. Uses the same AI_PROVIDER setting
# and API key (ANTHROPIC_API_KEY or GEMINI_API_KEY) as AI_DIAGNOSIS for the fallback path
Expand Down
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ Four project-scoped skills are available in Claude Code (stored in `.claude/skil

```
.env # credentials (git-ignored); see .env.example
.env.example # template: ORWELLSTAT_USER, ORWELLSTAT_PASSWORD, ENV, BASIC_AUTH_USER, BASIC_AUTH_PASSWORD, ANTHROPIC_API_KEY, GEMINI_API_KEY
.env.example # template: ORWELLSTAT_USER, ORWELLSTAT_PASSWORD, ENV, BASIC_AUTH_USER, BASIC_AUTH_PASSWORD, ANTHROPIC_API_KEY, GEMINI_API_KEY, OPENROUTER_API_KEY
.vars # CI repository variables (git-ignored); see .vars.example
.vars.example # template: CLAUDE_REVIEW, PLAYWRIGHT_TYPESCRIPT, BRUNO, QUALITY_METRICS, AI_DIAGNOSIS, AI_PROVIDER, AI_MODEL_FAST, AI_MODEL_STRONG, SELF_HEALING
.vars.example # template: AI_REVIEW, PLAYWRIGHT_TYPESCRIPT, BRUNO, QUALITY_METRICS, AI_DIAGNOSIS, AI_PROVIDER, AI_MODEL_FAST, AI_MODEL_STRONG, AI_REVIEW_PROVIDER, AI_REVIEW_MODEL, SELF_HEALING
.mcp.json # MCP server definitions (MCP_DOCKER, playwright, playwright-report-mcp) — loaded by Claude Code and other MCP-compatible AI assistants
.github/workflows/ # CI workflows (one per sub-project)
CLAUDE.md # repository-specific behavioral guidance for Claude Code
Expand Down Expand Up @@ -62,9 +62,10 @@ BASIC_AUTH_USER=<staging basic auth user>
BASIC_AUTH_PASSWORD=<staging basic auth password>
ANTHROPIC_API_KEY=<Anthropic API key>
GEMINI_API_KEY=<Gemini API key>
OPENROUTER_API_KEY=<OpenRouter API key (required when AI_REVIEW_PROVIDER=openrouter)>
```

`ORWELLSTAT_USER` and `ORWELLSTAT_PASSWORD` are required for all environments. `BASIC_AUTH_USER` and `BASIC_AUTH_PASSWORD` are only needed when running against staging — Playwright passes them automatically as HTTP Basic Auth credentials when set. `ANTHROPIC_API_KEY` and `GEMINI_API_KEY` are optional — when the active provider's key is present alongside `AI_DIAGNOSIS=true` in `.vars`, failed tests receive an `AI diagnosis` attachment in the Playwright report; when either is absent the fixture behaves identically to without them. Set `AI_PROVIDER` in `.vars` to choose the provider (`anthropic` by default, or `gemini`). In CI, secrets are injected as GitHub Actions secrets and variables via GitHub Actions variables. Sub-projects load them via `dotenv` with a path pointing two levels up (`../../.env` and `../../.vars`).
`ORWELLSTAT_USER` and `ORWELLSTAT_PASSWORD` are required for all environments. `BASIC_AUTH_USER` and `BASIC_AUTH_PASSWORD` are only needed when running against staging — Playwright passes them automatically as HTTP Basic Auth credentials when set. `ANTHROPIC_API_KEY` and `GEMINI_API_KEY` are optional — when the active provider's key is present alongside `AI_DIAGNOSIS=true` in `.vars`, failed tests receive an `AI diagnosis` attachment in the Playwright report; when either is absent the fixture behaves identically to without them. Set `AI_PROVIDER` in `.vars` to choose the provider (`anthropic` by default, or `gemini`). `OPENROUTER_API_KEY` is required when `AI_REVIEW_PROVIDER=openrouter` (the default for the PR reviewer); get one at https://openrouter.ai/keys. In CI, secrets are injected as GitHub Actions secrets and variables via GitHub Actions variables. Sub-projects load them via `dotenv` with a path pointing two levels up (`../../.env` and `../../.vars`).

### CI repository variables

Expand All @@ -76,7 +77,9 @@ The following variables are set in **GitHub → Settings → Variables → Actio
| `AI_PROVIDER` | AI provider for diagnosis (`anthropic` or `gemini`; default: `anthropic`) | Every Playwright run (local and CI) |
| `AI_MODEL_FAST` | Override fast-tier model (diagnosis); empty = provider default | Every Playwright run (local and CI) |
| `AI_MODEL_STRONG` | Override strong-tier model (selector fix); empty = provider default | Every Playwright run (local and CI) |
| `CLAUDE_REVIEW` | `claude-code-review.yml` | PR events (opened, synchronize, ready_for_review, reopened) |
| `AI_REVIEW_PROVIDER` | Provider for PR reviewer (`openrouter` default or `anthropic`) | PR events via `claude-code-review.yml` |
| `AI_REVIEW_MODEL` | Override reviewer model; empty = provider default (openrouter: `qwen/qwen3.6-plus`, anthropic: `claude-sonnet-4-6`) | PR events via `claude-code-review.yml` |
| `AI_REVIEW` | `claude-code-review.yml` | PR events (opened, synchronize, ready_for_review, reopened) |
| `PLAYWRIGHT_TYPESCRIPT` | `playwright-typescript.yml` | PR events, push to main, Sunday 03:00 UTC schedule, `workflow_dispatch` |
| `BRUNO` | `bruno.yml` | PR events, push to main, `workflow_dispatch` |
| `QUALITY_METRICS` | `quality-metrics.yml` | 1st of every month 06:00 UTC schedule, `workflow_dispatch` |
Expand Down
Loading