Skip to content

Compress optimization skills by 72%, add commit template #1711

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
blt wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Compress optimization skills by 72%, add commit template #1711

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
308 changes: 40 additions & 268 deletions .claude/skills/lading:optimize:hunt/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,303 +5,75 @@ description: Systematic optimization hunter for lading. Finds memory optimizatio

# Optimization Hunt

Systematically explores the lading codebase, implements optimizations, validates with benchmarks. **Finding bugs is equally valuable as finding optimizations.**
Find optimizations, benchmark them, record results. Bugs discovered during hunting are equally valuable.

## Valuable Outcomes
## Workflow

| Outcome | Value | Action |
|---------|-------|--------|
| **Optimization works** | Memory/time improved | Submit to `/lading:optimize:review` |
| **Optimization fails** | Learned cold path | Record, next target |
| **Bug discovered** | Found correctness issue | Invoke `/lading:optimize:validate` |
1. `/lading:preflight` → verify environment
2. Check `db.yaml` → skip already-attempted target/technique combos
3. Select target → profile or pick from hot paths
4. Implement ONE change on `opt/<crate>-<technique>` branch
5. Benchmark → micro then macro (see [BENCHMARKING.md](../shared/BENCHMARKING.md))
6. `ci/validate` → must pass
7. Commit with results → submit to `/lading:optimize:review`

**All three outcomes build institutional knowledge.**
## Target Selection

---

## Phase 0: Pre-flight

Run `/lading:preflight` first. Then check what's already been done:

```bash
# Check previous hunts
cat .claude/skills/lading:optimize:hunt/db.yaml

# Check previous reviews
cat .claude/skills/lading:optimize:review/db.yaml
```

**If a target/technique combination exists in either db.yaml, SKIP IT.**

---

## Phase 1: Select Target

### Target Sources

Use profiling data when available:
```bash
# CPU profiling (Mac)
sample <pid> 10 -file /tmp/profile.txt

# Or with samply (if installed)
samply record ./target/release/payloadtool ci/fingerprints/json/lading.yaml

# Memory profiling (use existing fingerprint configs)
./target/release/payloadtool ci/fingerprints/json/lading.yaml --memory-stats
```

**Note:** payloadtool takes a config YAML as its first argument. See `ci/fingerprints/*/lading.yaml` for examples. Config controls `seed`, `maximum_prebuild_cache_size_bytes`, and generator type.

Otherwise, check pending hunt issues or pick from hot subsystems:

| Crate | Hot Paths |
|-------|-----------|
| `lading_payload` | Block generation, cache, payload construction |
| `lading_throttle` | Capacity calculation, rate limiting |
| `lading` | Generators, blackholes, target management |

---

## Phase 2: Analyze Target

### Identify Opportunity Type
Hot paths: `lading_payload` (serialization), `lading_throttle` (rate math), `lading` (generators).

| Pattern | Technique | Bug Risk |
|---------|-----------|----------|
| `Vec::new()` + repeated push | `Vec::with_capacity(n)` | None |
| `String::new()` + repeated push | `String::with_capacity(n)` | None |
| `HashMap::new()` hot insert | `HashMap::with_capacity(n)` | None |
| Allocation in hot loop | Move outside loop | Lifetime issues |
| Clone where borrow works | Use reference | Lifetime complexity |
| Large struct by value | Box or reference | Nil/lifetime risk |
| Unbounded growth | Bounded buffer | Semantic change |
Profile with: `samply record ./target/release/payloadtool ci/fingerprints/<type>/lading.yaml`

**Watch for bugs while analyzing - they're valuable findings.**
## Benchmark Gates

### Lading-Specific Concerns
See [BENCHMARKING.md](../shared/BENCHMARKING.md) for commands.

- **Determinism**: Any optimization must preserve deterministic output
- **Pre-computation**: Prefer moving work to initialization over runtime
- **Worst-case behavior**: Optimize for worst-case, not average-case
**Pass if ANY threshold met:**
- Time: >=5% faster
- Memory: >=10% reduction
- Allocations: >=20% reduction

---

## Phase 3: Implement

```bash
git checkout main && git pull
git checkout -b opt/<crate>-<technique>
```

Make ONE change. Commit:
```bash
git commit -m "opt: <description>

Hypothesis: <expected improvement>
Technique: <prealloc|cache|avoid-clone|etc>
"
```

---

## Phase 4: Benchmark

**Two-stage gate: micro THEN macro. Both must show improvement.**
**Micro must pass before running macro.** If micro shows no improvement, record failure and move on.

### CRITICAL: Use Separate Worktree for Baseline
## Bug Discovery

**NEVER use `git stash`/`git checkout` to switch between baseline and optimized.** This causes confusion and errors. Instead, use a separate git worktree:
If you find a bug instead of an optimization: `/lading:optimize:validate`

```bash
# One-time setup: create a baseline worktree (do this once per repo)
git worktree add ../lading-baseline main
Then record as `bug_found` and continue hunting.

# The baseline worktree is at ../lading-baseline
# Your optimization work stays in the current directory
```

### Stage 1: Micro-benchmarks (inner loops)

Use `cargo criterion` for micro-benchmarks. Run in each worktree and compare output:
## Commit Format

```bash
# In baseline worktree (../lading-baseline)
cd ../lading-baseline
cargo criterion 2>&1 | tee /tmp/criterion-baseline.log
After benchmarks pass:

# In optimization worktree (your current directory)
cd /path/to/your/optimization/branch
cargo criterion 2>&1 | tee /tmp/criterion-optimized.log

# Compare results manually - look for "change:" lines showing improvement/regression
# Example output: "time: [1.2345 ms 1.2456 ms 1.2567 ms] change: [-5.1234% -4.5678% -4.0123%]"
```
<Verb> <target> <technique>

**Note:** Criterion automatically compares against the last run in that worktree and reports percentage changes.

#### Micro Decision Point

| Result | Action |
|--------|--------|
| Time improved >=5% | Proceed to Stage 2 |
| No change or regression | Record FAILURE, next target |
<Brief explanation of what and why>

**If micro-benchmark shows no improvement, STOP. Move to next target.**
**Micro-benchmark** (cargo criterion):
- <size>: <before> → <after> (<improvement>)

### Stage 2: Macro-benchmarks (end-to-end payloadtool)
**Macro-benchmark** (payloadtool --memory-stats):
- Memory: <before> → <after> (<-X%>)

Only run this if Stage 1 showed improvement.

```bash
# Choose a config file (e.g., ci/fingerprints/json/lading.yaml)
CONFIG=ci/fingerprints/json/lading.yaml

# In baseline worktree
cd ../lading-baseline
cargo build --release --bin payloadtool
hyperfine --warmup 3 --export-json /tmp/baseline.json \
"./target/release/payloadtool $CONFIG"
./target/release/payloadtool "$CONFIG" --memory-stats 2>&1 | tee /tmp/baseline-mem.txt

# In optimization worktree
cd /path/to/your/optimization/branch
cargo build --release --bin payloadtool
hyperfine --warmup 3 --export-json /tmp/optimized.json \
"./target/release/payloadtool $CONFIG"
./target/release/payloadtool "$CONFIG" --memory-stats 2>&1 | tee /tmp/optimized-mem.txt
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
```

#### Macro Decision Point
Title examples: `Reuse buffer in X`, `Eliminate allocations in Y`, `Use handle-based storage in Z`

| Result | Action |
|--------|--------|
| Time improved >=5% | Proceed to `/lading:optimize:review` |
| Memory reduced >=10% | Proceed to `/lading:optimize:review` |
| Allocations reduced >=20% | Proceed to `/lading:optimize:review` |
| No change or regression | Record FAILURE (micro win, macro loss), next target |
| **ci/validate fails** | Might be a **BUG** |
| **Determinism broken** | Might be a **BUG** |
## Recording

**All three gates must pass to proceed to review:**
1. Micro-benchmark shows improvement (>=5% time)
2. Macro-benchmark shows improvement (>=5% time OR >=10% memory OR >=20% allocations)
3. ci/validate passes (correctness preserved)

---
Update `db.yaml` in the SAME commit. See [DB_SCHEMA.md](../shared/DB_SCHEMA.md).

## Phase 5: Handle Bug Discovery

If during hunting you discover a bug (not an optimization):

### Invoke Correctness Validation
```
/lading:optimize:validate
```

This skill will:
1. Attempt Kani proof first (if feasible)
2. Fall back to property test if Kani fails
3. Verify fix works
4. Record in its db.yaml

### After Validation

Return here and record as BUG_FOUND in Phase 7, then **continue hunting** - don't stop.

---

## Phase 6: Validate (MANDATORY)

Before proceeding to review, ALL changes must pass:

```bash
ci/validate
```

**No exceptions. If ci/validate fails, fix the issue before continuing.**

### Kani Proofs (When Touching Critical Code)

If your optimization touches `lading_throttle` or `lading_payload`:

```bash
ci/kani lading_throttle # If throttle was modified
ci/kani lading_payload # If payload was modified
```

Kani is slow and may not compile complex code. If it fails:
1. Document why Kani couldn't run
2. Ensure comprehensive property tests exist instead

---

## Phase 7: Record & Continue

### MANDATORY: Update db.yaml

1. Add entry to `db.yaml` index
2. Create detailed file in `db/` directory

**db.yaml entry:**
```yaml
entries:
- target: <file:function>
technique: <prealloc|avoid-clone|cache|etc>
technique: <technique>
status: <success|failure|bug_found>
file: db/<target-technique>.yaml
```

**db/<target-technique>.yaml** for SUCCESS:
```yaml
target: <file:function>
technique: <prealloc|avoid-clone|cache|etc>
status: success
date: <YYYY-MM-DD>
branch: <branch name>
measurements:
time: <-X% or ~>
memory: <-X% or ~>
allocations: <-X% or ~>
lessons: |
<pattern learned>
file: db/<name>.yaml
```

**db/<target-technique>.yaml** for FAILURE:
```yaml
target: <file:function>
technique: <prealloc|avoid-clone|cache|etc>
status: failure
date: <YYYY-MM-DD>
reason: <cold path|already optimized|etc>
lessons: |
<why it didn't work - this is valuable knowledge>
```

**db/<target-technique>.yaml** for BUG_FOUND:
```yaml
target: <file:function>
technique: <what was being tried>
status: bug_found
date: <YYYY-MM-DD>
bug_description: <what was found>
validation_file: <path to validate db entry>
lessons: |
<what was learned>
```

### Immediately Continue

```
Target completed -> Back to Phase 1 -> Pick new target -> Never stop
```

---

## Usage

```
/lading:optimize:hunt
```
## Lading Rules

A 10% combined success rate (optimizations + bugs) is excellent. Most targets are cold paths - that's expected.
- **Determinism**: same seed → same output (always)
- **No panics**: no `.unwrap()` in non-test code
- **Kani**: run `ci/kani <crate>` if touching throttle/payload
Loading
Loading