Paper #2: 30-Day Regime Detection (Pivot from 5-Day Sequential)

[iAmGiG]

Critical Finding: 5-Day Sequential Approach Failed (Nov 5, 2025)

Discovery: 5-day trajectory analysis achieved 98-100% detection across ALL conditions (2020 weak GEX & 2024 strong GEX), confirming it detects universal dealer hedging (trivial) rather than distinctive patterns (interesting).

Root Cause:

• Dealers hedge options EVERY day (this is what Black-Scholes 1973 tells us)
• 5-day windows capture daily hedging noise, not regime structure
• 98-100% detection = no discrimination = no research contribution

Pivot Decision: Moving to 30-day regime windows to detect persistent structural patterns, not daily hedging flows.

---

New Research Question (30-Day Regime Framework)

Previous (5-day): "Can LLMs detect sequential GEX trajectories?"

• Answer: Yes, always (98-100% detection)
• Problem: Not research-worthy

New (30-day): "Can LLMs identify persistent market regimes from dealer gamma positioning?"

• Expected answer: Yes, selectively (30-50% detection)
• Why better: Meaningful discrimination, regime stability analysis

---

Methodology: Persistent Regime Detection

Regime Definition (30-Day Window)

A persistent regime exists when dealer constraints remain stable:

• Persistent Positive: >70% of days (21+/30) have positive net GEX, avg >\B
• Persistent Negative: >70% of days (21+/30) have negative net GEX, avg >\B

Non-Regimes (Rejected)

• Transitional: Frequent GEX sign flips, no dominant direction
• Low Conviction: Consistent sign but weak magnitude (<\B avg)

Classification Metrics

def classify_30day_window(gex_data):
    positive_days = count(gex > 0)  # Out of 30
    avg_magnitude = mean(abs(gex))
    sign_flips = count(sign changes)
    
    if positive_days >= 21 and avg_magnitude >= 5e9 and sign_flips <= 5:
        return "persistent_positive"  # DETECT
    elif negative_days >= 21 and avg_magnitude >= 5e9 and sign_flips <= 5:
        return "persistent_negative"  # DETECT
    else:
        return "transitional" or "low_conviction"  # REJECT

---

Expected Outcomes

Q1 2024 (Phase 1)

• Windows: 32 potential 30-day windows (61 trading days)
• Expected persistent regimes: 1-2 windows
• Expected detection rate: 60-80% (LLM detects the 1-2 persistent regimes)

Full 2024 (Phase 2)

• Windows: ~223 potential windows (252 trading days)
• Expected persistent regimes: 4-8 windows
• Expected detection rate: 30-50% (selective, meaningful)

2020 Comparison

• Expected persistent regimes: 2-4 windows (weaker pre-0DTE constraints)
• Expected detection rate: 20-30% (lower than 2024)
• Hypothesis: 0DTE proliferation → stronger regime persistence

---

Why This Strengthens the Research

5-Day Results (archived as negative finding):

• ✅ Validates methodology (LLM can detect patterns)
• ❌ But patterns are universal/trivial (98-100% always present)
• 📊 Will document as methodological exploration

30-Day Regime Approach:

• ✅ Detects persistent structural patterns (not daily noise)
• ✅ Expected 30-50% detection (meaningful selectivity)
• ✅ Enables 0DTE proliferation analysis (2020 vs 2024)
• ✅ Sets up Paper 🔴 Data Pipeline Foundation #3 (sector rotation at regime boundaries)

Research Contribution:

"LLM-based regime analysis identifies persistent dealer gamma constraints (30-day stability >70%) with 30-50% selectivity, distinguishing structural regimes from transitional periods. 0DTE proliferation (2020→2024) increased regime persistence by XX%."

---

Implementation

New Module

File: src/validation/regime_classifier.py (created)

• Classifies 30-day windows into regime types
• Deterministic ground truth for LLM validation
• Thresholds: 70% persistence, \B magnitude, ≤5 flips

Modified Files

• src/data_sources/sequential_gex_fetcher.py - Support window_size=30 parameter
• scripts/validation/validate_sequential_patterns.py - Regime validation
• src/llm/mechanics_prompt_builder.py - Regime detection prompt

---

Timeline

Week 1 (Nov 4-8): Implementation

• Design document created (docs/papers/paper2/methodology/regime_windows_design.md)
• RegimeClassifier module created
• Modify SequentialGEXFetcher for 30-day windows
• Design regime detection prompt
• Update validation script

Week 2 (Nov 11-15): Phase 1 + Phase 2

• Q1 2024 quick validation (32 windows)
• Full 2024 validation (~223 windows)
• Initial results analysis

Week 3 (Nov 18-22): Phase 3

• 2020 validation (comparison to 2024)
• 0DTE proliferation statistical analysis

Week 4 (Nov 25-29): Negative Controls

• Shuffled regime windows (should reject)
• Synthetic transitions (should reject)
• Low-magnitude persistent windows (should reject)

---

Documentation

Design Document: docs/papers/paper2/methodology/regime_windows_design.md
5-Day Archive: docs/papers/paper2/validation/test4/ (negative result, documented thoroughly)

---

Priority

CRITICAL - Core methodology pivot for Paper #2

Status: Implementation in progress

Next Steps:

1. Finish RegimeClassifier implementation
2. Modify SequentialGEXFetcher for 30-day support
3. Create regime detection prompt
4. Run Phase 1 Q1 2024 validation

---

Updated: November 5, 2025

[iAmGiG]

Paper #2/3 Direction Consolidated (Oct 25, 2025)

After reviewing all Paper #2/3 references across the repository, we've confirmed the advisor-suggested direction:

✅ Paper #2: Sequential GEX Analysis (Issue #89)

Research Question: Can LLMs detect constraint trajectories (not just snapshots)?

Methodology:

• 5-day lookback windows (Day T-4 to Day T+0)
• Detect temporal patterns: accumulation, relief, reversal, persistence
• Maintain obfuscation: "Day T-4", "Day T-3", etc.
• Compare single-day vs sequential detection

Timeline: Q1 2026 journal submission (after Paper #1 acceptance)

✅ Paper #3: Cross-Asset Generalization

Research Question: Does obfuscation testing generalize beyond SPY index options?

Methodology:

• Test on 10-20 individual stocks (high liquidity)
• Combine sequential analysis if Paper 🔴 Environment Setup & Configuration #2 validates it
• Compare index vs single-name dealer dynamics

Timeline: Q2 2026 journal submission

Superseded Ideas

Alpha decline investigation (Oct 13) → Fold into Paper #2 discussion section
Pattern discovery → Defer to Paper #4+ (different problem class)

Rationale: Advisor's sequential→equities path is most practical and scientifically sound. Uses existing data, clear contributions, natural progression.

See consolidated analysis: docs/research_roadmap.md (being created)