🤖 Autofix ✨

"In the ancient dojo of broken tests, where failed assertions echo like battle cries in the night, there walks a lone warrior. Armed with the swift blade of Claude AI and the ancient scrolls of code wisdom, Autofix strikes without mercy. One test. One fix. One path to honor. When your UI tests fall in disgrace, only one master can restore balance to the codebase. His weapon? Autonomous intelligence. His mission? Vengeance against bugs. His name is whispered in fear by broken assertions everywhere: Autofix - The Code Ronin."

A Toei Company Production · Directed by Kinji Fukasaku · Starring Claude-3.5-Haiku as The Silent Debugger

---

An autonomous AI agent that automatically fixes failing iOS UI tests using Claude AI and intelligent code analysis tools.

🎯 Overview

Autofix analyzes failed iOS UI tests, explores your codebase, and autonomously makes code changes to fix the failures. It can work in two modes depending on whether you want to fix your application code or your test code.

Key Features

• 🔍 Intelligent Test Analysis: Parses XCTest results and identifies failure details
• 🖼️ Visual Context: Analyzes simulator screenshots to understand UI state
• 🛠️ Autonomous Code Editing: Makes targeted code changes automatically
• ✅ Verification Loop: Builds and runs tests to verify fixes work
• 🎭 Dual Modes: Fix app code OR test code based on your needs
• 🤖 Multi-Provider Support: Use Claude, OpenAI, or local Ollama models
• 🔧 Tool-Based Architecture: Uses specialized tools for inspection, editing, and testing

📦 Installation

Prerequisites

• Rust (edition 2024)
• Xcode and xcodebuild command-line tools
• LLM Provider (choose one):
  • Claude (Anthropic) - Recommended, default
  • OpenAI (GPT-4, GPT-4o, etc.)
  • Ollama (Local models)

Build from Source

git clone <repository-url>
cd autofix
cargo build --release

LLM Provider Setup

Autofix supports three LLM providers. Choose the one that works best for you:

Option 1: Claude (Anthropic) - Default

Get your API key from console.anthropic.com

export ANTHROPIC_API_KEY="sk-ant-api03-..."
# Optional: Override default model
export AUTOFIX_MODEL="claude-sonnet-4"  # or claude-opus-4, claude-haiku-3.5

Option 2: OpenAI

Get your API key from platform.openai.com

export AUTOFIX_PROVIDER=openai
export OPENAI_API_KEY="sk-..."
# Optional: Override default model
export AUTOFIX_MODEL="gpt-4o"  # or gpt-4-turbo, gpt-4

OpenAI-Compatible Servers (Together.ai, Groq, vLLM, etc.):

export AUTOFIX_PROVIDER=openai
export AUTOFIX_API_BASE="https://your-server.com/v1"
export OPENAI_API_KEY="your-key"
export AUTOFIX_MODEL="your-model-name"

Option 3: Ollama (Local Models)

Install and start Ollama, then pull a model:

# Install Ollama from ollama.ai
ollama serve
ollama pull llama2  # or llama3, codellama, mistral, etc.

# Configure autofix
export AUTOFIX_PROVIDER=ollama
export AUTOFIX_MODEL="llama2"  # or your preferred model

Configuration File

Alternatively, create a .env file (see .env.example for all options):

cp .env.example .env
# Edit .env with your API keys and preferences

Rate Limiting

Autofix includes smart rate limiting to prevent hitting API limits:

# Maximum tokens per minute (provider-specific defaults)
export AUTOFIX_RATE_LIMIT_TPM=50000

Default limits by provider:

• Claude: 30,000 TPM
• OpenAI: 90,000 TPM
• Ollama: Unlimited (local)

🚀 Usage

Standard Mode (Fix Test Code)

Assumes your app is correct and the test needs adjustment:

# Using Claude (default)
autofix --ios \
  --test-result path/to/test.xcresult \
  --workspace path/to/workspace

# Using OpenAI
autofix --ios \
  --provider openai \
  --test-result path/to/test.xcresult \
  --workspace path/to/workspace

# Using Ollama (local)
autofix --ios \
  --provider ollama \
  --test-result path/to/test.xcresult \
  --workspace path/to/workspace

With verbose debug output:

autofix --ios \
  --test-result path/to/test.xcresult \
  --workspace path/to/workspace \
  --verbose

Override model:

autofix --ios \
  --provider openai \
  --model gpt-4o \
  --test-result path/to/test.xcresult \
  --workspace path/to/workspace

What it does:

• ✅ Analyzes test failures
• ✅ Fixes test code (selectors, waits, expectations)
• ✅ Adds accessibility identifiers to app code (for testability)
• ✅ Verifies fixes by running tests

Knight Rider Mode (Fix App Code)

Assumes your test is correct and the app needs fixing:

autofix --ios \
  --test-result path/to/test.xcresult \
  --workspace path/to/workspace \
  --knightrider

What it does:

• ✅ Treats test as source of truth
• ✅ Fixes application source code only
• ✅ Adds missing UI elements, labels, identifiers
• ✅ Never modifies test files

Verbose Mode

Add the -v or --verbose flag to any command to enable detailed debug output:

autofix --ios \
  --test-result path/to/test.xcresult \
  --workspace path/to/workspace \
  --verbose  # or -v

What verbose mode shows:

• File paths and directories being processed
• Test identifiers and metadata
• Tool execution details (operations, inputs, outputs)
• Token usage and rate limit statistics
• File sizes and content lengths
• Success/failure details for each operation

Note: AI conversation output is ALWAYS printed, regardless of verbose mode.

Test a Specific Test

Get detailed analysis for a single test:

autofix test --ios \
  --test-result path/to/test.xcresult \
  --workspace path/to/workspace \
  --test-id "test://com.apple.xcode/MyApp/MyTests/MyTests/testExample"

🎭 Mode Comparison

Mode	Assumption	Primary Target	Can Modify App?	Can Modify Test?
Standard (default)	App is correct	Fix test code	✅ Yes (accessibility)	✅ Yes
Knight Rider (--knightrider)	Test is correct	Fix app code	✅ Yes (only this)	❌ No

🛠️ How It Works

Architecture

Autofix uses a multi-stage pipeline:

1. Attachment Fetching: Extracts screenshots and attachments from .xcresult bundles
2. Test File Location: Finds the Swift test file in your workspace
3. AI Analysis: Claude analyzes the failure with visual context
4. Autonomous Fixing (with tools):
   • DirectoryInspectorTool: Explores codebase, reads files, searches for patterns
   • CodeEditorTool: Makes precise code edits via string replacement
   • TestRunnerTool: Builds and runs tests to verify fixes

Example Workflow

┌─────────────────────────────────────┐
│  Failed Test: testLoginButton()    │
│  Error: Button not found           │
└─────────────────────────────────────┘
              ↓
┌─────────────────────────────────────┐
│  🤖 Autofix analyzes screenshot     │
│  Sees button exists visually        │
└─────────────────────────────────────┘
              ↓
┌─────────────────────────────────────┐
│  🔍 Explores codebase               │
│  Finds LoginView.swift              │
└─────────────────────────────────────┘
              ↓
┌─────────────────────────────────────┐
│  ✏️ Adds accessibility ID            │
│  Button("Login")                    │
│    .accessibilityIdentifier("...")  │
└─────────────────────────────────────┘
              ↓
┌─────────────────────────────────────┐
│  🧪 Runs test                       │
│  ✅ Test passes!                    │
└─────────────────────────────────────┘

📂 Project Structure

autofix/
├── src/
│   ├── main.rs                          # CLI entry point
│   ├── llm/                             # LLM provider abstraction
│   │   ├── mod.rs                       # Core types & factory
│   │   ├── provider_trait.rs            # LLMProvider trait
│   │   ├── config.rs                    # Provider configuration
│   │   ├── claude_provider.rs           # Claude/Anthropic impl
│   │   ├── openai_provider.rs           # OpenAI impl
│   │   └── ollama_provider.rs           # Ollama impl
│   ├── pipeline/                        # Core pipeline logic
│   │   ├── mod.rs                       # Module declarations
│   │   ├── autofix_pipeline.rs          # Pipeline implementation
│   │   └── prompts.rs                   # AI prompt generation
│   ├── tools/                           # AI agent tools
│   │   ├── directory_inspector_tool.rs  # File exploration
│   │   ├── code_editor_tool.rs          # Code editing
│   │   └── test_runner_tool.rs          # Build & test execution
│   ├── autofix_command.rs               # Process all failed tests
│   ├── test_command.rs                  # Single test processing
│   ├── rate_limiter.rs                  # Provider-aware rate limiting
│   ├── xcresultparser.rs                # Parse XCResult bundles
│   ├── xctestresultdetailparser.rs      # Parse test details
│   ├── xc_test_result_attachment_handler.rs  # Extract attachments
│   └── xc_workspace_file_locator.rs     # Locate test files
├── Cargo.toml
├── .env.example                         # Configuration template
└── README.md

🔧 Tools

Autofix provides the LLM with three specialized tools:

DirectoryInspectorTool

• Operations: list, read, search, find
• Purpose: Explore workspace, read files, search for patterns
• Example: Find all Swift files with a specific class

CodeEditorTool

• Operation: Exact string replacement
• Purpose: Make targeted code edits
• Safety: Validates old content exists before replacing

TestRunnerTool

• Operations: build, test
• Purpose: Compile code and run specific tests
• Output: Exit codes, stdout, stderr for verification

📊 Example Output

🤖 Knight Rider iteration 1...

💭 Claude says:
I'll explore the codebase to understand the app structure and locate the relevant view files.

🔧 Tool call: directory_inspector (id: toolu_123)
   Input: {"operation": "list", "path": "MyApp"}

🔧 Tool call: directory_inspector (id: toolu_456)
   Input: {"operation": "read", "path": "MyApp/Views/LoginView.swift"}

🤖 Knight Rider iteration 2...

💭 Claude says:
I found the issue. The button exists but lacks an accessibility identifier.

🔧 Tool call: code_editor (id: toolu_789)
   Input: {...}
   ✏️ Edit result: Successfully edited file: MyApp/Views/LoginView.swift

🔧 Tool call: test_runner (id: toolu_abc)
   Input: {"operation": "test", "test_identifier": "..."}
   🧪 Test result: Test passed (exit code: 0)
   ✅ SUCCESS!

✓ Knight Rider finished!

🧪 Development

Run Tests

cargo test

Run with Debug Logging

RUST_LOG=debug cargo run -- --ios --test-result ... --workspace ...

Build for Release

cargo build --release
./target/release/autofix --help

🎯 Common Use Cases

1. Missing Accessibility Identifiers

Problem: Test can't find UI elements Solution: Autofix adds .accessibilityIdentifier() to views

2. Incorrect Test Selectors

Problem: Test uses wrong element query Solution: Autofix updates test to use correct selector

3. Timing Issues

Problem: Test fails due to animation/loading Solution: Autofix adds proper wait conditions

4. Wrong Assertions

Problem: Test expects incorrect text/state Solution: Autofix updates test assertions

5. Missing UI Elements

Problem: App missing button/label test expects Solution: (Knight Rider mode) Autofix adds missing elements to app

⚠️ Limitations

• iOS/Xcode projects only (Android support planned)
• Requires xcodebuild command-line tools
• Works best with structured, well-named code
• May need multiple iterations for complex fixes
• Requires valid API key for Claude/OpenAI, or local Ollama setup

🤝 Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests: cargo test
5. Submit a pull request

📄 License

GPL-3.0

🙏 Acknowledgments

• Built with support for multiple LLM providers:
  • Anthropic Claude via anthropic-sdk-rust
  • OpenAI via async-openai
  • Ollama for local model support
• Inspired by the need for better UI test maintenance

---

Made with ❤️ and 🤖 AI