Pattern Taxonomy: Focus on Core Mechanical Patterns

[iAmGiG]

Problem

We have 15 patterns but need to focus on the core 5-7 patterns that are:

1. Mechanically validated (must happen due to dealer constraints)
2. Economically significant (survive transaction costs)
3. High conviction (>60% success rate with sufficient samples)

Core Patterns to Validate

Tier 1: Academically Proven Mechanical Patterns

These have academic research proving they MUST occur:

1. Gamma Positioning

   • Academic: Buis et al. 2024
   • Mechanism: Delta-neutral mandate forces hedging
   • Success Rate: 72% (from testing)

2. Stock Pinning (OPEX Pin)

   • Academic: Jeannin et al. 2008
   • Mechanism: Gamma explosion at strikes forces pin
   • Success Rate: 75% (from testing)

3. 0DTE Delta Hedging

   • Academic: Recent 0DTE papers
   • Mechanism: 40-50% SPX volume forces immediate hedging
   • Observable: Strike breach cascades

Tier 2: High Conviction Patterns (Need Validation)

These show strong results but need mechanical proof:

4. Gamma Squeeze

   • Success Rate: 67%
   • Mechanism: Call buying forces dealer hedge cascade
   • Need: Obfuscation test to prove it's not narrative

5. Friday 3:30 PM Effects

   • Success Rate: 75%
   • Mechanism: Final hedging window before expiration
   • Need: Test without temporal context

Patterns to Deprioritize

Too complex or low conviction:

• Window Dressing (52% success)
• Dispersion Trade (42% success)
• Correlation Breakdown (48% success)
• Earnings Straddle (45% success)
• Quarter-End Rebalancing (54% success)

Validation Framework

The Key Test: Obfuscation

Pattern is REAL if it works when the LLM doesn't know:

• The date/time (Friday 3:30 PM)
• The ticker (GME, SPY)
• The event context (OPEX, FOMC)

Success Criteria

• Mechanical: Clear causal mechanism why it MUST happen
• Success Rate: >60% with 30+ samples
• Economic: >20bps after costs
• Persistent: <10% annual alpha decay

Implementation Steps

1. Focus Testing on Core 5 Patterns

   • Run obfuscation tests using existing data_obfuscation.py
   • Calculate economic significance after transaction costs
   • Document causal mechanisms in pattern_taxonomy.py

2. Create Dealer State Machine

   Market State → Constraint → Forced Action → Pattern
   

3. Validate Against Baseline

   • Use Issue Baseline Without LLM: Raw GEX Strategy Performance #58 (Baseline Without LLM)
   • Prove patterns beat random

Deliverables

1. Validated Core Pattern Set (5-7 patterns max)
2. Mechanical vs Narrative Classification
3. Trading Rules for Validated Patterns Only

Key Insight

"The State Machine Reality: Dealers are constrained to limited actions. This IS a state machine with predictable transitions. The papers confirm dealers MUST hedge when gamma exposure exceeds risk limits."

Technical Implementation

• Framework: src/validation/pattern_taxonomy.py (already created)
• Integration: Leverage existing obfuscation system
• Testing: Use existing validation infrastructure

Related Issues

• Baseline Without LLM: Raw GEX Strategy Performance #58 - Baseline Without LLM (critical for validation)
• Implement 30-Day Forward Testing Protocol #49 - 30-Day Forward Testing
• Expand Testing Sample Size and Coverage (Priority 1) #43 - Expand Testing Sample Size

Issues Closed

• ✅ Six-Category Pattern Detection & Regime Classification System #31 - Six-Category System (duplicate)
• ✅ Dynamic Prompt Generation System for Context-Aware LLM Analysis #32 - Dynamic Prompt Generation (premature)
• ✅ LLM Few-Shot Training Pipeline #38 - Few-Shot Training (premature)

[iAmGiG]

Phase 1 Validation Complete ✅

Test Period: Q1 2024 (January 2 - March 29, 2024)
Sample Size: 53 trading days
Method: Obfuscation testing (no date/ticker context to LLM)

---

Results Summary

Pattern	Success Rate	Samples	Classification	Verdict
Gamma Positioning	100%	53/53	✅ MECHANICAL	Validated
Stock Pinning	100%	53/53	✅ MECHANICAL	Validated
0DTE Hedging	100%	53/53	✅ MECHANICAL	Validated
Dealer Trap	37.7%	20/53	⚠️ PROBABILISTIC	Not validated
Friday 3:30 PM	0%	0/53	❌ FOLKLORE	Failed obfuscation
Volume Anomaly	0%	0/53	❌ FOLKLORE	Failed obfuscation

Status: ⚠️ PARTIAL SUCCESS (3/6 patterns validated, target was 5-7)

---

Validated Patterns (100% Success)

All 3 patterns detected correctly on every test date without knowing dates, tickers, or market context:

1. Gamma Positioning (Buis et al. 2024)

• WHO → WHOM → WHAT: Option buyers → Dealers → Forced delta hedging
• Constraint: Delta-neutral regulatory mandate
• Detection: LLM identified forced hedging flows from gamma exposure alone

2. Stock Pinning (Jeannin et al. 2008)

• WHO → WHOM → WHAT: Large OI concentrations → Dealers → Price gravitates to strikes
• Constraint: Gamma explosion at high-OI strikes
• Detection: LLM identified pinning mechanics from OI distributions

3. 0DTE Hedging (Academic support)

• WHO → WHOM → WHAT: 0DTE traders → Dealers → Rapid hedging
• Constraint: Time decay acceleration (40-50% SPX volume in 0DTE)
• Detection: LLM identified immediate hedging requirements

---

Failed Patterns

Dealer Trap (37.7% - Probabilistic)

• Below 60% threshold
• Inconsistent detection across dates
• No clear mechanical constraint

Friday 3:30 PM (0% - Folklore)

• Issue: Temporal dependency (requires knowing day/time)
• Obfuscation test strips this context
• LLM cannot detect without time labels

Volume Anomaly (0% - Folklore)

• No clear causal mechanism
• Pure statistical anomaly
• No consistent WHO → WHOM → WHAT

---

What This Proves

✅ The LLM detects structural mechanics: All 3 patterns with clear constraints (delta-neutral mandate, gamma explosion, time decay) were detected 100% of the time without memorization.

❌ Temporal/narrative patterns fail: Patterns requiring calendar knowledge (Friday 3:30) or lacking mechanisms (volume anomaly) cannot be detected from data alone.

⚠️ The State Machine Reality: Validated patterns exploit situations where dealers have no choice:

• Must hedge or violate risk limits (Gamma)
• Must rehedge constantly or face gamma explosion (Pinning)
• Must hedge immediately or compound exponential risk (0DTE)

---

Deliverables

Documentation:

• Full summary: reports/validation/ISSUE_79_VALIDATION_SUMMARY.md
• Individual results: reports/validation/pattern_taxonomy/*.yaml (6 files)
• Batch summary: reports/validation/pattern_taxonomy/all_patterns_summary_20251002_181921.yaml

Validation Scripts:

• Single pattern: scripts/validate_pattern_taxonomy.py
• Batch validator: scripts/validate_all_patterns.py

---

Next Steps (Phase 2)

Criteria remaining:

• ⏳ Economic Significance: Calculate returns after transaction costs (>20bps required)
• ⏳ Baseline Comparison: Prove LLM adds value over raw GEX strategy (Issue Baseline Without LLM: Raw GEX Strategy Performance #58)

Recommended Actions:

1. ✅ Proceed with 3 validated patterns to economic backtest
2. ✅ Deploy only validated patterns to production
3. ⚠️ Consider dealer_trap as probabilistic edge (test in Phase 2)
4. ❌ Discard friday_330_squeeze and volume_anomaly as folklore
5. 📊 Optional: Expand validation to Q2-Q4 2024 or additional symbols (QQQ, IWM)

Commands for Phase 2:

# Economic backtest
python scripts/run_baseline_strategy.py \
  --patterns gamma_positioning stock_pinning 0dte_hedging \
  --start-date 2024-01-02 \
  --end-date 2024-03-29 \
  --calculate-returns

[iAmGiG]

File Organization & Cleanup Complete

Reorganization:

• ✅ Validation scripts moved to scripts/validation/ (no longer at root level)
• ✅ Documentation moved to docs/guides/pattern-validation.md (follows project structure)
• ✅ YAML naming convention updated: {pattern}_{TICKER}_{daterange}.yaml

Files Removed (89 total):

• Removed reports/experiments/ (64 intermediate test outputs)
• Removed quick test runs (11 YAMLs with partial data)
• Removed debug files and duplicates

Final Results (7 validation files):

• gamma_positioning_SPY_2024Q1.yaml (100% success, 53/53)
• stock_pinning_SPY_2024Q1.yaml (100% success, 53/53)
• 0dte_hedging_SPY_2024Q1.yaml (100% success, 53/53)
• dealer_trap_SPY_2024Q1.yaml (37.7%, 20/53)
• friday_330_squeeze_SPY_2024Q1.yaml (0%, 0/53)
• volume_anomaly_SPY_2024Q1.yaml (0%, 0/53)
• all_patterns_summary_SPY_2024Q1.yaml (batch summary)

Code Updates:

• validate_pattern_taxonomy.py: Filename generation uses ticker + date range (no timestamps)
• validate_all_patterns.py: Updated script paths and summary naming
• Documentation updated: todo.md, CLAUDE.md, docs/README.md

Ready for Phase 2 (economic backtest).