Thalesians Blade Runner

An auditable, ISO-friendly experiment runner:

• Per-run UUID+UTC timestamp directories
• Git checkout with exact SHAs logged
• Stdout/stderr capture with tee
• Configurable metric extraction (regex/JSON/text)
• Markdown + HTML (optional PDF) reports with embedded figures
• Cryptographic signing of manifests and reports
• Parameter grid and random search
• Optional upload to S3 (WORM/immutability via bucket policies + object lock)

Note

Pre-release; work in progress. Use at your own risk.

Install

pip install .[report-html,pdf,s3]   # extras are optional

Quick start

blade-runner init-config > blade_runner.example.json
blade-runner run --config blade_runner.example.json --out ./blade_runs

Features behind extras

• report-html: pretty HTML reports using Jinja2
• pdf: render HTML to PDF via WeasyPrint if available
• s3: push run folders or selected artifacts to S3

Compliance notes (ISO/MiFID/Model Risk)

Controls scaffolding provided:

• Full run manifest with checksums (integrity) and environment snapshot (traceability)
• Git SHAs for source provenance (configuration control)
• Immutable session/run layout suitable for WORM/object-lock storage
• Optional signatures for tamper-evidence
• Parameter configs stored alongside outputs for reproducibility

To align with ISO 9001/27001 and MiFID II/RTS 6 expectations:

• Store session directories in an immutable tier (S3 Object Lock, on-prem WORM)
• Enforce code review for config changes; record change tickets in governance
• Maintain retention schedules and data classification for artifacts
• Capture approvals and deployment checks referencing produced reports/manifests

Demo: local git repo + sample config

Below is a tiny, self-contained demo wired to a local git repo (no external dependencies). It simulates a "training" script that prints metrics, writes a JSON metrics file, a text report, and an SVG figure. Blade Runner then extracts those and builds the report.

1) Create a minimal repo

demo_repo/setup.sh

#!/usr/bin/env bash
set -euo pipefail

# Create a tiny demo repo with a training script
rm -rf demo_repo
mkdir -p demo_repo/artifacts
cat > demo_repo/train.py << 'PY'
#!/usr/bin/env python3
import argparse, json, math, os, random, time
from pathlib import Path

parser = argparse.ArgumentParser()
parser.add_argument('--lr', type=float, default=1e-3)
parser.add_argument('--epochs', type=int, default=5)
args = parser.parse_args()

random.seed(42)
Path('artifacts').mkdir(exist_ok=True)

losses = []
base = 0.5 + 0.5*math.exp(-max(args.lr,1e-8)*10)
for e in range(1, args.epochs+1):
    # fake decreasing loss with some noise
    val = base / e + random.random()*0.01
    losses.append(val)
    print(f"epoch={e} train_loss={val:.6f}")
    time.sleep(0.02)

# Write JSON metrics
metrics = {
    "validation": {"auc": max(0.5, 1.0 - losses[-1])},
    "final": {"loss": losses[-1]}
}
(Path('artifacts')/ 'metrics.json').write_text(json.dumps(metrics, indent=2), encoding='utf-8')

# Write a text summary we can scrape
(Path('artifacts')/ 'report.txt').write_text(f"RMSE: {losses[-1]*1.2:.6f}
", encoding='utf-8')

# Produce a tiny SVG figure (no external libs)
svg = [
    '<svg xmlns="http://www.w3.org/2000/svg" width="400" height="200">',
    '<rect width="100%" height="100%" fill="#ffffff"/>',
    '<polyline fill="none" stroke="#1f77b4" stroke-width="2" points="'
]
# map losses to y (invert)
maxy = max(losses); miny = min(losses)
pts = []
for i, v in enumerate(losses):
    x = 20 + i*(360/(max(1,len(losses)-1)))
    # normalize to [20, 180]
    y = 180 - ((v - miny)/max(1e-9, (maxy-miny))) * 160
    pts.append(f"{x:.1f},{y:.1f}")
svg.append(' '.join(pts))
svg.append("' />")
svg.append('<text x="20" y="20" font-family="monospace" font-size="12">loss</text>')
svg.append('</svg>')
(Path('artifacts')/ 'loss.svg').write_text('
'.join(svg), encoding='utf-8')

print("done")
PY
chmod +x demo_repo/train.py

# Init git repo
cd demo_repo
git init -q
git checkout -b main -q
git add .
git -c user.email=test@example.com -c user.name="Blade Demo" commit -qm "Initial demo commit"
cd ..

# Print the local file:// URL for config convenience
ABS=$(cd demo_repo && pwd)
echo "Local repo URL: file://$ABS"

2) Sample Blade Runner config

blade_runner.demo.json

{
  "run_root": "./blade_runs",
  "git_repos": [
    { "url": "file://REPLACE_WITH_ABSOLUTE_PATH/demo_repo", "ref": "main", "dest": "demo_repo" }
  ],
  "workdir": "./", 
  "command": "python train.py",
  "env": {},
  "capture": { "tee": true, "stdout_file": "stdout.log", "stderr_file": "stderr.log" },
  "extract": {
    "stdout_patterns": [
      { "name": "final_train_loss", "regex": "train_loss\s*=\s*([0-9.]+)\$?", "group": 1, "type": "float" }
    ],
    "stderr_patterns": [],
    "json_files": [
      { "name": "auc", "path": "artifacts/metrics.json", "json_path": "validation.auc" },
      { "name": "loss_json", "path": "artifacts/metrics.json", "json_path": "final.loss" }
    ],
    "text_files": [
      { "name": "rmse", "path": "artifacts/report.txt", "regex": "RMSE: ([0-9.]+)", "group": 1, "type": "float" }
    ]
  },
  "report": {
    "title": "Blade Runner Report — Demo",
    "fields": ["status", "return_code", "final_train_loss", "auc", "rmse"],
    "images": ["artifacts/loss.svg"],
    "html": true
  },
  "artifacts": {
    "include": ["artifacts", "logs/stdout.log", "logs/stderr.log", "reports/report.md"],
    "exclude_glob": ["**/__pycache__/**"]
  },
  "search": {
    "grid": { "LR": [0.001, 0.01], "EPOCHS": [5, 10] },
    "cli_template": "{command} --lr {LR} --epochs {EPOCHS}",
    "env_prefix": "BR_",
    "max_runs": 3
  },
  "governance": {
    "run_owner": "paul.bilokon@imperial.ac.uk",
    "change_ticket": "CHG-DEMO-0001",
    "purpose": "Demo backtest/training run",
    "risk_category": "R&D",
    "data_inputs": ["synthetic"],
    "policies": ["SDLC-001", "INFOSEC-002"]
  },
  "signing": { "enable": false },
  "upload": { "s3": { "enable": false, "bucket": "", "prefix": "", "object_lock": false } }
}

Replace REPLACE_WITH_ABSOLUTE_PATH with the absolute path echoed by setup.sh (e.g., /home/you/path).

3) Run the demo

# 1) Create the local git repo
bash demo_repo/setup.sh

# 2) Edit the config to point to your local file:// path
$EDITOR blade_runner.demo.json

# 3) Execute with HTML (and PDF/signing if you enabled extras)
blade-runner run \
  --config blade_runner.demo.json \
  --out ./blade_runs \
  --html

What you’ll see

• Per-run folders under blade_runs/session_*/run_*/
• logs/stdout.log capturing lines like epoch=5 train_loss=...
• artifacts/metrics.json, artifacts/report.txt, artifacts/loss.svg
• reports/report.md and reports/report.html
• manifest.json with checksums, repo SHAs, env snapshot
• index.json summarising up to 3 grid runs (as configured)