Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 179 additions & 0 deletions EXP-020_LIVE_VALIDATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
# EXP-020 Live Validation Template

> **Purpose**: Structured go/no-go checklist for validating EXP-020 (fee-aware gating + session metrics) in production before enabling real capital.
> **Duration**: Run for **60–120 minutes** minimum before passing judgment.

---

## 1. Pre-Launch Checks

| # | Check | Command / Action | Status |
|---|-------|-----------------|--------|
| 1 | Bot starts without crash | `npm run dev:bot` — watch for startup logs | ☐ |
| 2 | Inventory snapshot logged | Look for `[INVENTORY] startup config` + first `[INVENTORY] snapshot` | ☐ |
| 3 | Rebalance loop started | Look for `[INVENTORY] starting rebalance loop` | ☐ |
| 4 | Session summary timer started | No explicit log for this — confirmed by first `[SESSION] periodic_summary` (after `summaryIntervalMs`) | ☐ |
| 5 | Executor speed-tier resolved | Look for `[SOLANA] speed_tier_resolved` with `tier`, `maxLamports`, `priorityLevel` | ☐ |
| 6 | Priority fee estimator active | Look for `[SOLANA] fee_estimate` with `source: 'helius'` (or `'static-config'` if Helius unavailable) | ☐ |

---

## 2. Execution Funnel Validation (per `[SESSION] periodic_summary`)

Wait for the first periodic summary (default: 10 min), then capture these fields:

| Field | Expected Range | Red Flag | Value |
|-------|---------------|----------|-------|
| `discovered` | > 0 after 10 min | 0 → scanner isn't finding opportunities | |
| `gateEvaluated` | ≤ `discovered` | > discovered → counter bug | |
| `gatePassed` | ≤ `gateEvaluated` | 0 with many evaluated → gates too tight | |
| `gateRejected` | 0–100% of evaluated | 100% sustained → review reject reasons | |
| `swapsBuilt` | ≥ `gatePassed` | < gatePassed → build failures | |
| `submitted` | ≤ `swapsBuilt` | 0 → submission never reached | |
| `confirmed` | > 0 eventually | 0 after 30+ trades → landing issue | |
| `expired` | < 20% of submitted | > 50% → fee too low or RPC lag | |
| `failed` | < 10% of submitted | > 30% → contract/slippage issue | |

### Reject Reason Breakdown

Capture from `reject_*` fields in the summary log:

| Reason Key | Meaning | Concern If |
|-----------|---------|-----------|
| `reject_net_edge_negative` | Fee exceeds gross profit | Dominant → fees too high for current spreads |
| `reject_low_landing_rate` | Landing rate below threshold | Frequent → RPC/fee calibration needed |
| `reject_profit_below_floor` | Net profit below `minNetEdgeUsd` | OK if frequent early — spreads narrow |
| `reject_accumulator_negative` | Trailing net-edge window negative | Protective circuit breaker, should be rare |

---

## 3. Fee & Economics Validation

### Per-Trade Logs

Capture from each `[SOLANA] execution fee` log:

| Field | Expected | Red Flag |
|-------|----------|----------|
| `feeLamports` | 10,000–500,000 | > 1M consistently → fee estimator miscalibrated |
| `executionFeeUsd` | $0.001–$0.05 (typical) | > $0.10 → high, review speed tier |
| `netExpectedAfterFeesUsd` | > 0 | < 0 → gate failed to block unprofitable trade |
| `solPriceUsd` | ~current SOL price | 0 → InventoryManager price feed broken |

### Session Summary Averages

| Field | Healthy Range | Red Flag |
|-------|--------------|----------|
| `avgFeeBpsOfNotional` | 5–100 bps | > 200 bps → fees eating too much notional |
| `avgNetEdgeBpsOfNotional` | > 0 bps | < 0 sustained → losing money after fees |
| `avgExpectedGrossUsd` | > 0 | Null → no confirmed trades with fee data |
| `avgExecutionFeeUsd` | $0.001–$0.05 | > $0.10 → review |
| `avgNetEdgeUsd` | > 0 | < 0 → systemic loss |

---

## 4. Quote Freshness

| Field | Healthy | Red Flag |
|-------|---------|----------|
| `avgQuoteAgeMs` | < 2000ms | > 5000ms → stale quotes, likely slippage |
| `minQuoteAgeMs` | < 500ms | N/A |
| `maxQuoteAgeMs` | < 10000ms | > 15000ms → some quotes extremely stale |

Also check per-submit log `[SOLANA] tx_submitted` for `quoteAgeMs` on individual trades.

---

## 5. Landing Rate

From `[SOLANA] landing_report` (logged periodically by LandingTracker):

| Field | Healthy | Red Flag |
|-------|---------|----------|
| `landingRate` | > 0.5 (50%) | < 0.3 → fee tier too low |
| `windowConfirmed` | > 0 | 0 after 10+ submits → serious issue |
| `windowExpired` | < 30% of submitted | > 50% → blockheight/timing issue |
| `consecutiveFailures` | 0–2 | > 5 → auto-escalation should kick in |

---

## 6. Rebalance Gate

Check `[INVENTORY] rebalance_gate` logs and session summary:

| Field | Expected | Red Flag |
|-------|----------|----------|
| `rebalanceEvaluated` | > 0 (if auto-fund on) | 0 after 60 min → rebalance loop not triggering |
| `rebalanceRejected` | < 50% of evaluated | 100% → `maxRebalanceCostBps` too tight |
| `costBps` in gate log | < 100 (default max) | Consistently near max → fees volatile |

---

## 7. Go / No-Go Decision Matrix

After **60 minutes of observation**, evaluate:

| Criterion | PASS | FAIL | Your Result |
|-----------|------|------|-------------|
| Bot running without crashes | Uptime > 60 min | Any crash/restart | |
| Opportunities discovered | `discovered > 0` | 0 after 60 min | |
| Gate passing some trades | `gatePassed > 0` | 0 with `gateEvaluated > 10` | |
| Landing rate acceptable | > 30% | < 15% over window | |
| Net edge positive | `avgNetEdgeUsd > 0` or null (no confirms yet) | `avgNetEdgeUsd < 0` sustained | |
| Fee BPS reasonable | `avgFeeBpsOfNotional < 200` | > 300 sustained | |
| Quote freshness OK | `avgQuoteAgeMs < 5000` | > 10000 sustained | |
| No accumulator circuit break | No `reject_accumulator_negative` dominance | > 80% of rejections | |
| Rebalance functioning | Gate evaluating (if auto-fund on) | 0 evaluations after 60 min | |

### Decision:
- **All PASS**: Proceed to EXP-021 (Jito bundles) planning
- **1–2 FAIL (non-critical)**: Adjust config (fees, thresholds), re-run validation
- **3+ FAIL or any crash**: Diagnose root cause before proceeding

---

## 8. Config Knobs to Adjust If Needed

| Env Var | Current Default | Adjust If |
|---------|----------------|-----------|
| `MIN_NET_EDGE_USD` | 0.001 | Too many `reject_profit_below_floor` → lower |
| `MAX_REBALANCE_COST_BPS` | 100 | Rebalances always rejected → raise to 150–200 |
| `SPEED_TIER` | `medium` | Landing rate < 30% → try `aggressive` |
| `PRIORITY_FEE_PERCENTILE` | 75 | Fees too high → lower to 50; too low landing → raise to 90 |
| `LANDING_RATE_FLOOR` | 0.25 | Too many `reject_low_landing_rate` → lower |
| `SESSION_SUMMARY_INTERVAL_MS` | 600000 | Want faster feedback → set to 120000 (2 min) |
| `NET_EDGE_WINDOW_SIZE` | 20 | Accumulator triggering too fast → raise to 50 |
| `NET_EDGE_MIN_SAMPLES` | 5 | Not enough data for verdict → lower to 3 |

---

## 9. Log Grep Quick Reference

```bash
# Periodic session summary
grep "periodic_summary" bot.log

# All gate rejections with reasons
grep "execution_gate" bot.log | grep "passed.*false"

# Fee logs for confirmed trades
grep "execution fee" bot.log

# Quote age on specific submits
grep "tx_submitted" bot.log

# Landing rate reports
grep "landing_report" bot.log

# Rebalance gate decisions
grep "rebalance_gate" bot.log

# Net-edge accumulator state
grep "net_edge_snapshot" bot.log
```

---

**Template Version**: EXP-020.1
**Created**: 2026-04-13
**Next Experiment**: EXP-021 (Jito bundles) — blocked on EXP-020 green validation
54 changes: 54 additions & 0 deletions env.example
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,60 @@ SOLANA_PRIVATE_KEY_BASE58=
# Valid: governance_compromise, code_exploit, oracle_attack, insider, mev
# SOLANA_RISK_DENY_INCIDENT_TYPES=governance_compromise

# --- Inventory Manager / Auto-Funding ---
# Enable automatic SOL reserve + base-asset rebalancing. Default: false.
# SOLANA_AUTO_FUND_ENABLED=false
# Base trading asset — excess SOL and non-base stables are converted to this. USDC | USDT.
# SOLANA_BASE_ASSET=USDC
# Minimum SOL reserve kept for rent+fees (never sold below this). Default: 0.15.
# SOLANA_MIN_SOL_RESERVE=0.15
# Target SOL amount after a top-up swap. Default: 0.25.
# SOLANA_TARGET_SOL_RESERVE=0.25
# Minimum swap size for funding rebalances (USD). Skips tiny swaps. Default: 0.50.
# SOLANA_AUTO_FUND_MIN_SWAP_USD=0.50
# Interval between periodic rebalance checks (ms). Default: 60000 (60s).
# SOLANA_FUNDING_REBALANCE_INTERVAL_MS=60000
# Fraction of available capital used per trade. Default: 0.25.
# SOLANA_POSITION_SIZE_FRACTION=0.25
# Floor per-trade size (USD). Default: 0.20.
# SOLANA_MIN_TRADE_USD=0.20
# Ceiling per-trade size (USD). Default: 250.
# SOLANA_MAX_TRADE_USD=250
# Re-invest realized profits into trading capital. Default: true.
# SOLANA_COMPOUND_PROFITS=true

# ── Dynamic Priority Fee ──
# Enable dynamic fee estimation via getRecentPrioritizationFees. Default: true.
# SOLANA_DYNAMIC_PRIORITY_FEE=true
# Percentile of recent slot fees to target (0-100). Default: 75.
# SOLANA_PRIORITY_FEE_PERCENTILE=75
# Minimum maxLamports floor (0.00001 SOL). Default: 10000.
# SOLANA_PRIORITY_FEE_FLOOR_LAMPORTS=10000
# Maximum maxLamports cap (0.005 SOL). Default: 5000000.
# SOLANA_PRIORITY_FEE_CAP_LAMPORTS=5000000
# Static fallback maxLamports when dynamic lookup fails. Default: 1000000.
# SOLANA_PRIORITY_FEE_STATIC_LAMPORTS=1000000
# Cache TTL for fee estimates (ms). Default: 10000.
# SOLANA_PRIORITY_FEE_CACHE_TTL_MS=10000

# ── EXP-020: Fee-aware execution gating ──────────────────────────────
# Speed tier: standard | priority | ultra. Default: standard.
# SOLANA_SPEED_TIER=standard
# Minimum net profit after fees + slippage + risk buffer (USD). Default: 0.10.
# SOLANA_MIN_NET_PROFIT_USD=0.10
# Risk buffer deducted from gross edge (USD). Default: 0.05.
# SOLANA_RISK_BUFFER_USD=0.05
# Fallback slippage cost when quote lacks otherAmountThreshold (USD). Default: 0.02.
# SOLANA_SLIPPAGE_FALLBACK_USD=0.02
# Max rebalance cost in bps before skipping funding swap. Default: 100.
# SOLANA_AUTO_FUND_MAX_REBALANCE_COST_BPS=100
# Landing rate threshold below which auto-escalation activates. Default: 0.70.
# SOLANA_LANDING_RATE_WARNING_THRESHOLD=0.70
# Enable auto-escalation of priority fee percentile on low landing rate. Default: true.
# SOLANA_LANDING_RATE_AUTO_ESCALATE=true
# Rolling window size for net-edge accumulator. Default: 20.
# SOLANA_NET_EDGE_WINDOW=20

# Alerts Configuration
ALERT_MIN_CONFIDENCE=0.8
ALERT_TELEGRAM_TOKEN=your_telegram_bot_token
Expand Down
Loading
Loading