AI-Powered Automated Stock Trading System

Version 2.0 - Adaptive Ensemble Edition

Overview

Sophisticated automated trading system featuring a Hybrid Ensemble Architecture combining machine learning and reinforcement learning for the Korean stock market (KOSPI/KOSDAQ).

Performance Highlights (Samsung Electronics, 2020-2023):

• 📈 16.48% cumulative return (4.12% annualized)
• 📊 Sharpe Ratio: 1.54 (Excellent risk-adjusted returns)
• 🛡️ Max Drawdown: -9.39% (Strong risk control)

Core Components:

• XGBoost: Supervised learning on technical indicators (11.46% return)
• RL Agent (PPO): Adaptive policy optimization via reinforcement learning
• Adaptive Ensemble: Intelligent dynamic weight allocation (16.48% return)

---

Key Features

Hybrid AI Architecture

• XGBoost classifier with triple barrier labeling
• PPO agent with clipped surrogate objective
• Adaptive Ensemble with performance-based dynamic weighting (Phase 2 upgrade)

Advanced Optimizations

• Relaxed thresholds for increased trading opportunities (±0.15, XGBoost 0.48/0.52)
• Stop-Loss (-2%) and Take-Profit (+5%) for risk management
• Realistic transaction costs (0.0075%)
• Smart regime filtering (2-out-of-3 conditions)

Feature Engineering

• 10 RL-optimized indicators (price ratios, normalized momentum, volatility)
• Macro integration (USD/KRW, SOX, VIX)
• Vectorized triple barrier (5-10x speedup)

RL Environment

• Custom Gym-compatible simulation
• 13-dimensional state space (10 market + 3 position features)
• Discrete action space: {Sell, Hold, Buy}
• Realistic transaction modeling (fees, slippage)

Evaluation

• Unified evaluation script
• 3-way comparison (XGBoost vs RL vs Ensemble)
• Automated visualization

---

Project Structure

ai-automated-trade/
├── src/
│   ├── data/              # Data pipeline and preprocessing
│   ├── features/          # Technical indicators
│   ├── models/            # XGBoost, Ensemble
│   ├── rl/                # PPO agent, environment, training
│   ├── execution/         # Backtesting
│   └── utils/             # Configuration
├── evaluate_models.py     # Unified evaluation script
├── tests/                 # Unit tests
├── archive/               # Deprecated scripts
├── config.yaml
└── requirements.txt

---

Installation

# Clone and setup
git clone <repository-url>
cd ai-automated-trade
python -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Configure
cp .env.example .env
# Edit .env with your API credentials

---

Usage

Unified Evaluation

# 3-way comparison on single stock
python evaluate_models.py --model all --tickers AAPL

# Multiple stocks with visualization
python evaluate_models.py --model all --tickers AAPL,MSFT,GOOGL --visualize

# Custom configuration
python evaluate_models.py --model all --tickers 005930.KS \
    --start 2020-01-01 --end 2023-12-31 --rl-timesteps 100000

Options:

• --model: {xgboost, rl, ensemble, all}
• --tickers: Comma-separated symbols
• --start / --end: Date range
• --visualize: Generate charts
• --rl-timesteps: Training duration

Output:

• results/evaluation_results.json
• results/evaluation_summary.png

Recommended Usage (Adaptive Ensemble)

The Adaptive Ensemble mode automatically adjusts model weights based on recent performance, providing the best risk-adjusted returns.

# Single stock evaluation (Adaptive Ensemble is default)
python evaluate_models.py --model ensemble --tickers 005930.KS --rl-timesteps 100000

# Multiple stocks with visualization
python evaluate_models.py --model all --tickers 005930.KS,000660.KS,035420.KS --visualize

# Compare all models
python evaluate_models.py --model all --tickers 005930.KS --rl-timesteps 100000

Expected Performance:

• Ensemble: ~15-17% (4-year cumulative)
• XGBoost: ~10-12%
• RL: ~5-7% (higher variance)

Train Individual Models

XGBoost:

from src.models.numeric_model import NumericModel
model = NumericModel(model_path='models/xgb.json')
model.train(df, target_col='target')

RL:

# 100K timesteps recommended for stability
python -m src.rl.train --mode conservative --timesteps 100000

Ensemble:

from src.models.ensemble import EnsemblePredictor

# Adaptive mode (recommended)
ensemble = EnsemblePredictor(
    xgb_path='models/xgb.json',
    rl_path='models/ppo',
    mode='adaptive',  # Dynamic weight allocation
    alpha=0.5  # Initial weight for XGBoost
)

signal, info = ensemble.predict(df)
print(f"Signal: {signal}, Current alpha: {info['alpha']:.2f}")

---

Technical Details

Reinforcement Learning Formulation

State Space ($\mathbf{s}_t \in \mathbb{R}^{13}$):

$$ \mathbf{s}_t = \begin{bmatrix} \text{price_sma_ratio} \\ \text{rsi_norm} \\ \text{atr_pct} \\ \vdots \\ \text{current_position} \\ \text{days_in_position} \\ \text{unrealized_pnl} \end{bmatrix} $$

Action Space ($a_t \in {0, 1, 2}$):

$$ a_t = \begin{cases} 0 & \text{Sell/Short} \\ 1 & \text{Hold/Flat} \\ 2 & \text{Buy/Long} \end{cases} $$

Reward Function:

$$ R_t = \alpha \cdot \text{Sharpe}_t + (1-\alpha) \cdot \text{Return}_t $$

where:

$$ \text{Sharpe}_t = \frac{\mu_r}{\sigma_r} \sqrt{252}, \quad \text{Return}_t = \mu_r \times 252, \quad \alpha = 0.7 $$

PPO Algorithm

Clipped Surrogate Objective:

$$ L^{\text{CLIP}}(\theta) = \mathbb{E}_t \left[ \min \left( r_t(\theta) \hat{A}_t, , \text{clip}(r_t(\theta), 1-\epsilon, 1+\epsilon) \hat{A}_t \right) \right] $$

where:

$$ r_t(\theta) = \frac{\pi_\theta(a_t | s_t)}{\pi_{\theta_{\text{old}}}(a_t | s_t)}, \quad \epsilon = 0.2 $$

Value Function Loss:

$$ L^{\text{VF}}(\theta) = \mathbb{E}_t \left[ (V_\theta(s_t) - V_t^{\text{target}})^2 \right] $$

Total Objective:

$$ L^{\text{TOTAL}}(\theta) = \mathbb{E}_t \left[ L^{\text{CLIP}}(\theta) - c_1 L^{\text{VF}}(\theta) + c_2 H(\pi_\theta) \right] $$

where $c_1 = 0.5$, $c_2 = 0.01$

Hyperparameters:

• Learning rate: $3 \times 10^{-4}$
• Discount factor: $\gamma = 0.99$
• GAE parameter: $\lambda = 0.95$
• Clip range: $\epsilon = 0.2$
• Batch size: 64
• Training steps: 2048

Ensemble Strategy

Fixed Mode:

$$ \text{signal}_{\text{ensemble}} = \alpha \cdot \text{signal}_{\text{xgb}} + (1-\alpha) \cdot \text{signal}_{\text{rl}} $$

Adaptive Mode (dynamic weight allocation):

$$ \text{score}_{\text{xgb}} = \max(0.1, \text{perf}_{\text{xgb}} + 0.1), \quad \text{score}_{\text{rl}} = \max(0.1, \text{perf}_{\text{rl}} + 0.1) $$

$$ \alpha_t = \frac{\text{score}_{\text{xgb}}}{\text{score}_{\text{xgb}} + \text{score}_{\text{rl}}}, \quad \text{where } \text{perf} = \mathbb{E}[\text{returns}_{t-20:t}] $$

This gives minimal weight to negative-performing models and maximizes weight to the better performer.

Regime-Based Mode:

$$ \alpha = \begin{cases} 0.3 & \text{if } P_t > \text{SMA}_{200} \quad (\text{bull market}) \\ 0.7 & \text{otherwise} \quad (\text{bear market}) \end{cases} $$

---

Performance Metrics

Backtest Results (2020-2023, Samsung Electronics 005930.KS)

Adaptive Ensemble achieved 16.48% cumulative return (4.12% annualized), significantly outperforming individual models:

Model	Return	Sharpe Ratio	Max Drawdown	Status
Adaptive Ensemble	16.48%	1.54	-9.39%	⭐ Recommended
XGBoost	11.46%	1.20	-7.77%	✅ Stable
RL (PPO)	5.41%	0.55	-20.13%	⚠️ Volatile
Buy & Hold	23.34%	-	-	Benchmark

Performance Comparison

graph LR
    A["XGBoost<br/>11.46%"] -->|Ensemble<br/>Synergy| C["Adaptive Ensemble<br/>16.48%<br/>(+44% boost)"]
    B["RL Agent<br/>5.41%"] -->|Ensemble<br/>Synergy| C
    
    style A fill:#ffd93d
    style B fill:#ff9999
    style C fill:#4ecdc4

Loading

Key Features & Optimizations

Advanced Trading Logic:

• Adaptive Ensemble: Dynamic weight allocation based on recent model performance
• Smart Thresholds: Optimized buy/sell signals (XGBoost: 0.48/0.52, Ensemble: ±0.15)
• Risk Management: Automated Stop-Loss (-2%) and Take-Profit (+5%)
• Regime Filtering: Multi-factor market condition analysis (2-out-of-3 criteria)

Technical Optimizations:

• Realistic transaction costs (0.0075%)
• Extended RL training (100K timesteps)
• Performance-based weight adjustment every 20 days

Sharpe Ratio Comparison

Model	Sharpe	Rating
Ensemble	1.54	Excellent (>1.5)
XGBoost	1.20	Very Good (>1.0)
RL (PPO)	0.55	Fair (0.5-1.0)

Sharpe Ratio Scale: <0.5 (Poor), 0.5-1.0 (Fair), 1.0-1.5 (Good), >1.5 (Excellent)

Risk-Adjusted Performance

graph TD
    A[Adaptive Ensemble] --> B[High Return: 16.48%]
    A --> C[Excellent Sharpe: 1.54]
    A --> D[Low Drawdown: -9.39%]
    
    B --> E[Best Risk-Adjusted Returns]
    C --> E
    D --> E
    
    style A fill:#4ecdc4
    style E fill:#6bcf7f

Loading

Model Characteristics

Characteristic	XGBoost	RL (PPO)	Ensemble
Stability	⭐⭐⭐⭐⭐	⭐⭐	⭐⭐⭐⭐⭐
Profitability	⭐⭐⭐⭐	⭐⭐⭐	⭐⭐⭐⭐⭐
Risk Control	⭐⭐⭐⭐⭐	⭐⭐	⭐⭐⭐⭐⭐
Adaptability	⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐⭐

Key Insights

✅ Adaptive Ensemble: Best overall performance (16.48%, Sharpe 1.54)

• Dynamically allocates weight to better-performing models
• Reduces impact of RL volatility
• Creates synergy effect (ensemble > sum of parts)

✅ XGBoost: Most stable individual model (11.46%, Sharpe 1.20)

• Consistent performance across conditions
• Low drawdown (-7.77%)
• Strong technical pattern recognition

⚠️ RL (PPO): High variance (5.41%, Sharpe 0.55)

• Sensitive to training hyperparameters
• Requires more data for stability
• Best used as ensemble component, not standalone

---

Testing

# Run all tests
python -m pytest tests/ -v

# Specific test
python -m pytest tests/test_ensemble.py -v

# With coverage
python -m pytest tests/ --cov=src

---

Configuration

Edit config.yaml:

data:
  ticker: "005930.KS"
  start_date: "2020-01-01"
  end_date: "2023-12-31"

labeling:
  pt_pct: 0.02  # Profit target
  sl_pct: 0.01  # Stop loss

risk:
  initial_capital: 10000000

models:
  rl:
    learning_rate: 0.0003
    n_steps: 2048
    gamma: 0.99

---

Dependencies

Core libraries:

• stable-baselines3 (PPO)
• gymnasium (RL environment)
• xgboost (Gradient boosting)
• pandas-ta (Technical indicators)
• yfinance (Data)

See requirements.txt for full list.

---

References

Reinforcement Learning:

• Schulman et al. (2017). "Proximal Policy Optimization Algorithms" [arXiv:1707.06347]
• Sutton & Barto (2018). "Reinforcement Learning: An Introduction"

Financial ML:

• López de Prado (2018). "Advances in Financial Machine Learning"

---

Disclaimer

FOR EDUCATIONAL AND RESEARCH PURPOSES ONLY

This software is provided for educational purposes. Trading carries significant financial risk. The authors assume no responsibility for financial losses. Always backtest thoroughly and use paper trading before live deployment.

---

Last Updated: 2026-01-26 | Version: 2.0.1 - Adaptive Ensemble Edition
Best Performance: Adaptive Ensemble (16.48%, Sharpe 1.54)