Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
fdebbd9
Initialize BNPL production deployment v0.1.0
whitehackr Sep 26, 2025
69bbf13
Implement single-transaction feature engineering for production API
whitehackr Sep 26, 2025
130d635
Add unknown category monitoring for production readiness
whitehackr Sep 27, 2025
ec6d84c
Implement flexible multi-model predictor with calibration handling
whitehackr Sep 27, 2025
c656481
Add Claude Code assistant configuration and documentation standards
whitehackr Sep 27, 2025
0221c60
Implement production-ready BNPL API endpoints
whitehackr Sep 27, 2025
5edb87d
Reorganize test structure with proper unit/integration separation
whitehackr Sep 27, 2025
d735f04
Add ML data infrastructure architecture with MLflow integration
whitehackr Sep 27, 2025
c069f5c
Add Shadow Mode Controller design document
whitehackr Sep 27, 2025
0228163
Implement Shadow Mode Controller with storage abstraction
whitehackr Sep 27, 2025
3a37b7d
Add comprehensive unit tests for Shadow Mode Controller
whitehackr Sep 27, 2025
2b7bdc7
Add integration tests for Shadow Mode Controller with real ML components
whitehackr Sep 27, 2025
cf359f7
Add Redis storage implementation for production deployment
whitehackr Sep 27, 2025
0c118f6
Add Redis environment file security to gitignore
whitehackr Sep 27, 2025
bbd0d7e
Add Redis configuration template for development
whitehackr Sep 27, 2025
cc292c0
Add Redis dependency for production deployment
whitehackr Sep 27, 2025
98d983a
Implement Redis integration and production storage factory
whitehackr Sep 27, 2025
d2917d2
Integrate Shadow Controller with BNPL API endpoints
whitehackr Sep 27, 2025
d4f8361
Add python-dotenv dependency for cleaner env file handling
whitehackr Sep 27, 2025
618e4c3
Simplify Redis configuration with python-dotenv and Railway focus
whitehackr Sep 27, 2025
fa327a4
Add Dockerfile for Railway production deployment
whitehackr Sep 27, 2025
6d1e8b3
Add MLflow integration for experiment tracking and model registry
whitehackr Sep 27, 2025
6e74a7e
Add Railway deployment config and production environment template
whitehackr Sep 27, 2025
e00ff6a
Add MLflow artifacts to gitignore and document limitations
whitehackr Sep 27, 2025
b5ccaa6
Fix Railway deployment with requirements.txt fallback
whitehackr Sep 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .claude/settings.local.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"permissions": {
"allow": [
"Bash(git add:*)"
],
"deny": [],
"ask": []
}
}
43 changes: 43 additions & 0 deletions .env.production.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Production Environment Configuration Template for Railway Deployment
# These environment variables should be set in Railway's environment settings

# Redis Configuration (automatically set by Railway Redis service)
# REDIS_URL=redis://... # Railway sets this automatically

# ML Prediction Storage Configuration
ML_PREDICTION_TTL=2592000
ML_STORAGE_VERBOSE=false

# MLflow Configuration (local development - ephemeral in Railway)
MLFLOW_TRACKING_URI=sqlite:///mlflow.db
MLFLOW_EXPERIMENT_NAME=bnpl_shadow_mode_production

# API Configuration
PORT=8000

# Application Environment
ENVIRONMENT=production

# Logging Configuration
LOG_LEVEL=INFO

# Performance Tuning
UVICORN_WORKERS=1
UVICORN_MAX_REQUESTS=1000
UVICORN_TIMEOUT_KEEP_ALIVE=5

# Shadow Mode Configuration
SHADOW_MODE_ENABLED=true
DEFAULT_DECISION_POLICY=balanced

# Notes for Railway Deployment:
# 1. Railway automatically sets REDIS_URL when Redis service is added
# 2. PORT is automatically set by Railway
# 3. All other variables should be set in Railway's environment settings
# 4. MLflow data will be stored in the container filesystem (ephemeral)
# 5. For persistent MLflow storage, configure external MLflow server
#
# Known Limitations (to be addressed in future PR):
# - MLflow data is ephemeral and disappears on container restart
# - No team access to MLflow UI - only local development
# - Consider hosted MLflow server for production team collaboration
21 changes: 21 additions & 0 deletions .env.redis.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Redis Configuration for ML Prediction Caching - Flit Project
#
# INSTRUCTIONS:
# 1. Copy this file to .env.redis
# 2. Replace YOUR_REDIS_URL_HERE with actual Redis URL from Railway
# 3. Never commit .env.redis to git
#
# Get Redis URL from Railway:
# 1. Go to railway.app -> flit project -> Redis service -> Variables tab
# 2. Copy REDIS_URL value
# 3. Paste below

REDIS_URL=YOUR_REDIS_URL_HERE
REDIS_DB_TRANSACTIONS=0
REDIS_DB_PREDICTIONS=1
REDIS_TTL_SECONDS=604800
RAILWAY_PROJECT=flit

# BigQuery Configuration
BIGQUERY_PROJECT=flit-data-platform
GOOGLE_APPLICATION_CREDENTIALS=/../../../../.gcp/your-gcp-credentials.json
10 changes: 10 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,12 @@
__pycache__/
logs/

# Environment files (contain sensitive credentials)
.env
.env.*
!.env.*.template

# MLflow artifacts (local development only)
mlflow.db
mlruns/
mlflow-artifacts/
104 changes: 104 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Claude Code Assistant Configuration

## Documentation Standards

### PR Comment Technical Depth Requirements

All PR comments documenting technical decisions must meet **Sr Principal Engineer** standards:

#### Content Depth
- **Educational First**: Write to teach junior, mid, and senior engineers simultaneously
- **Technical Rigor**: Explain not just what was built, but why design decisions were made
- **Trade-off Analysis**: Document alternatives considered and rejection rationale
- **Context Setting**: Establish problem space before presenting solutions
- **Probabilistic Thinking**: When applicable, explain statistical/ML reasoning in depth

#### Writing Style
- **Prose Over Bullets**: Use solid prose for complex explanations; bullets only for simple lists
- **Human Voice**: Sound like an experienced engineer teaching, not AI-generated content
- **No Attribution**: Never reference "Sr Principal" titles, experience levels, or self-promotion. Also, no attributions to Claude Code.
- **Depth Without Repetition**: Be comprehensive but never redundant

#### Technical Documentation Pattern
```markdown
## Problem Context
[Establish the technical challenge and why it matters]

### Design Decision: [Clear title]
[Explain the chosen approach with code examples]

### Alternative Approaches Considered
[Detail what was rejected and why]

### Technical Deep Dive
[Explain the underlying principles, statistics, algorithms]

### Trade-offs & Future Considerations
[Long-term implications and improvement paths]
```

#### Code Quality Standards
- Always show working code examples
- Include performance metrics and benchmarks
- Document edge cases and error handling
- Explain statistical/mathematical foundations when relevant

### Development Commands

Standard commands for this repository:

#### Testing
```bash
# Run all tests (organized)
python run_tests.py

# Run specific test categories
poetry run pytest tests/unit/ -v # Unit tests only
poetry run pytest tests/integration/ -v # Integration tests only

# Run individual test modules
python tests/unit/features/test_feature_engineering.py
python tests/unit/models/test_multi_model_predictor.py
python tests/integration/test_api_endpoints.py
```

#### Linting & Type Checking
```bash
# Run before committing
poetry run ruff check .
poetry run mypy .
```

#### Model Development
```bash
# Activate environment
poetry shell

# Production artifact validation
poetry run pytest tests/integration/ -v
```

### Commit Message Standards

- **Concise but complete**: Summarize the change and key technical points
- **No excessive detail**: Technical depth belongs in PR comments, not commit messages
- **Imperative mood**: "Implement X" not "Implemented X"
- **Reference documentation**: When complex decisions are made, reference PR comments

### Known Issues Management

Maintain `docs/models/domain_monthyr_known_issues.md` with:
- **Clear prioritization**: 🔴 Blocking, 🟡 Important, 🟢 Enhancement
- **Technical solutions**: Specific code examples for fixes
- **Business impact**: How issues affect production deployment
- **Timeline guidance**: When to address each issue

### Educational Philosophy

Every technical document should enable a junior/mid/sr engineer to:
1. **Understand the problem** being solved
2. **Learn the reasoning** behind design decisions
3. **Implement similar solutions** in different contexts
4. **Recognize trade-offs** in their own work

This documentation serves as both project record and engineering education resource.
35 changes: 35 additions & 0 deletions DEPLOYMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# BNPL Production Deployment v0.1.0

## Phase 2: Core Pipeline Development

Production deployment of BNPL ML models with shadow mode capabilities.

### Current Implementation Status

- [x] Production branch setup
- [x] Artifact validation (6/6 models ready)
- [ ] Single-transaction feature engineering
- [ ] Multi-model predictor
- [ ] API endpoints
- [ ] Shadow mode controller
- [ ] Docker + Railway deployment

### Target Architecture

```
API Request → Feature Engineering → Multi-Model Prediction → Shadow Logging → Business Decision
```

### Performance Goals

- <100ms transaction processing (from 2ms research baseline)
- 36 features exactly matching training pipeline
- Support for 4 model deployment modes

## Next Steps

1. Implement `engineer_single_transaction()` method
2. Create flexible `BNPLPredictor` class
3. Build REST API endpoints
4. Add MLflow integration
5. Docker containerization for Railway
48 changes: 48 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# BNPL ML API Production Dockerfile for Railway Deployment
FROM python:3.11-slim

# Set environment variables for production
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PIP_NO_CACHE_DIR=1 \
PIP_DISABLE_PIP_VERSION_CHECK=1

# Set working directory
WORKDIR /app

# Install system dependencies for ML libraries
RUN apt-get update && apt-get install -y \
build-essential \
curl \
&& rm -rf /var/lib/apt/lists/*

# Poetry configuration (commented out due to Railway build issues)
# RUN pip install poetry==1.6.1
# ENV POETRY_NO_INTERACTION=1 \
# POETRY_VENV_IN_PROJECT=1 \
# POETRY_CACHE_DIR=/tmp/poetry_cache
# COPY pyproject.toml poetry.lock ./
# RUN poetry install --only=main && rm -rf $POETRY_CACHE_DIR

# Use requirements.txt for Railway deployment (workaround for Poetry connection issues)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY . .

# Create non-root user for security
RUN useradd --create-home --shell /bin/bash app \
&& chown -R app:app /app
USER app

# Expose port (Railway will set PORT environment variable)
EXPOSE 8000

# Health check for Railway
HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
CMD curl -f http://localhost:$PORT/v1/bnpl/health || exit 1

# Start command for Railway deployment (updated for pip-based install)
# CMD poetry run uvicorn flit_ml.api.main:app --host 0.0.0.0 --port $PORT
CMD python -m uvicorn flit_ml.api.main:app --host 0.0.0.0 --port ${PORT:-8000}
Loading