Poser: Advanced Skier Pose Analysis Pipeline

Poser is a full-stack video analysis system for skier pose detection, turn segmentation, and technique metrics. It ships as a web app (React + Rails) and a GPU-enabled analysis service.

Visual Example

Landing Page	Results Dashboard

Key Features

• End-to-end web workflow: Upload, analyze, and review results from a browser with live status updates.
• Embeddable partner widget: Hosted JS widget with domain allowlisting and email confirmation flow.
• Pose analysis pipeline: YOLO26 + BoT-SORT tracking, MediaPipe 3D pose, and temporal smoothing.
• Technique metrics: Edge similarity scoring and turn segmentation surfaced in the UI.
• Artifact exports: Pose overlay video and CSV metrics for parity checks.
• GPU acceleration: CUDA on Linux and MPS on Apple Silicon, with a native-analysis workflow for Macs.

Documentation

• Pipeline details and metrics: docs/analysis-pipeline.md
• Web app spec: docs/spec-webapp.md
• Embed widget flow: docs/embed-widget.md
• Rails backend details: backend/README.md

Processing Pipeline (High Level)

1. Load video metadata and normalize rotation.
2. Detect skiers and estimate 3D pose landmarks.
3. Smooth landmarks in time.
4. Compute metrics (edge similarity + turn segmentation).
5. Render outputs and publish artifacts.

See docs/analysis-pipeline.md for the full breakdown.

Repository Structure

poser/
├── analysis/                 # Analysis code (service + pipeline)
│   ├── service/              # FastAPI analysis service + scripts
│   └── core/                 # Analysis pipeline (runner + step modules)
│       └── src/              # Step implementations (video_loading, pose_detection, temporal_smoothing, metrics_calculation, output_preparation)
├── backend/                  # Rails API server (auth, storage, jobs)
│   ├── app/                  # Controllers, services, serializers
│   ├── db/                   # Migrations + structure.sql
│   └── spec/                 # Rails specs
├── frontend/                 # React + Vite frontend + embed widget
│   ├── src/                  # UI, pages, embed widget
│   └── dist/                 # Built assets (generated)
├── docs/                     # Product + engineering docs
├── docker-compose.yml        # Local dev services
├── fly.web.toml              # Fly.io config for poser-web
├── fly.analysis.toml         # Fly.io config for poser-analysis
├── Caddyfile.local           # Local reverse proxy rules
├── tests/                    # Repo-level tests (e.g., Caddyfile)
├── input/                    # Local sample inputs
├── output/                   # Local analysis outputs
└── scratch/                  # Local experiments

Development

Docker Compose (local stack)

Run the web app, database, and proxy locally:

docker compose up

Start the analysis container as well:

COMPOSE_PROFILES=analysis docker compose up

Note: local uploads require S3 credentials in .env (Tigris or an S3-compatible bucket).

Local URLs:

• App: http://localhost
• Backend: http://localhost:8000

Native analysis on Apple Silicon (GPU acceleration)

CUDA is not supported in Docker on Macs, so run the analysis service on the host for MPS:

export ANALYSIS_SERVICE_URL=http://host.docker.internal:8080
docker compose up -d db web caddy

source analysis/service/setup_native_env.sh
python -m analysis.service.server

Configuration

The analysis defaults live in analysis/core/configs/default.yaml and include inline comments. Edit that file to tweak device selection, model paths, and smoothing values.

Container Architecture

The same topology is used locally and on Fly; analysis runs in a container or on the host, depending on your setup.

Browser
  │
  ▼
Edge Proxy
  - Local: Caddy (`Caddyfile.local`)
  - Production: Fly edge router
  │
  ▼
poser-web (Rails + static frontend)
  │  ├─ reads/writes: poser-db (Postgres)
  │  ├─ presigned uploads/downloads: Tigris S3
  │  └─ triggers: poser-analysis (/analyze)
  │
  ▼
poser-analysis (GPU service)
  ├─ downloads input from Tigris S3
  ├─ runs analysis/service/analyze.py
  └─ posts progress + artifacts to /api/internal/* on poser-web

Public endpoints (poser-web)

• POST /api/auth/request-code
• POST /api/auth/verify-code
• POST /api/analyses/create-upload
• POST /api/analyses/{id}/confirm-upload
• GET /api/analyses
• GET /api/analyses/{id}
• GET /api/analyses/{id}/edge-similarity
• GET /api/analyses/{id}/turns
• GET /api/analyses/{id}/download/{artifact_kind}
• DELETE /api/analyses/{id}
• POST /api/contact
• GET /api/embed/{partner_slug}/config
• POST /api/embed/{partner_slug}/submit
• POST /api/embed/{partner_slug}/upload-complete
• GET /api/embed/{partner_slug}/status/{analysis_id}
• GET /api/embed/{partner_slug}/feedback/{analysis_id}
• GET /api/embed/confirm?token=...
• GET /api/embed/results/{token}

Internal endpoints (analysis → backend)

• PUT /api/internal/analyses/{id}/progress
• PUT /api/internal/analyses/{id}/status
• PUT /api/internal/analyses/{id}/edge-similarity
• POST /api/internal/analyses/{id}/turns
• POST /api/internal/artifacts

Database Schema

┌──────────────────────────────┐         ┌──────────────────────────────┐
│            users             │         │      verification_codes      │
├──────────────────────────────┤         ├──────────────────────────────┤
│ id (PK)                      │         │ id (PK)                      │
│ email (unique)               │◀────────│ email                        │
│ verified_at                  │         │ code (bcrypt hash)           │
│ created_at                   │         │ expires_at                   │
│ updated_at                   │         │ used                         │
└──────────────────────────────┘         │ created_at                   │
              │                          └──────────────────────────────┘
              │ 1:N
              ▼
┌──────────────────────────────┐         ┌──────────────────────────────┐
│           analyses           │         │            uploads           │
├──────────────────────────────┤         ├──────────────────────────────┤
│ id (PK)                      │         │ id (PK)                      │
│ user_id (FK)                 │◀────────│ user_id (FK)                 │
│ partner_id (FK)              │         │ analysis_id (FK)             │
│ status                       │────────▶│ key (unique)                 │
│ filename                     │         │ size, mime, sha256           │
│ progress (JSON)              │         │ status                       │
│ analysis_results (JSON)      │         │ created_at, updated_at       │
│ edge_similarity (JSONB)      │         └──────────────────────────────┘
│ edge_similarity_* (float)    │
│ turns (JSONB)                │         ┌──────────────────────────────┐
│ s3_input_key                 │         │           artifacts          │
│ trim_start_seconds           │         ├──────────────────────────────┤
│ trim_end_seconds             │         │ id (PK)                      │
│ confirmed_at                 │         │ analysis_id (FK)             │
│ error_log                    │         │ key (unique)                 │
│ created_at, updated_at       │         │ kind, size, mime, sha256      │
└──────────────────────────────┘         │ created_at                   │
              │ 1:N                    ▲ └──────────────────────────────┘
              ▼                        │
┌──────────────────────────────┐        │
│           partners           │        │
├──────────────────────────────┤        │
│ id (PK)                      │        │
│ user_id (FK)                 │────────┘
│ slug (unique)                │
│ domain                       │
│ created_at, updated_at       │
└──────────────────────────────┘

┌──────────────────────────────┐
│    embed_confirmation_tokens │
├──────────────────────────────┤
│ id (PK)                      │
│ user_id (FK)                 │
│ analysis_id (FK)             │
│ token_hash (unique)          │
│ expires_at                   │
│ used                         │
│ created_at                   │
└──────────────────────────────┘

Production & CI/CD

Fly.io deployment

• poser-web: Combined backend + frontend (static assets built into the image).
• poser-analysis: GPU analysis service (auto-start/auto-stop).
• poser-db: Fly Postgres cluster attached to poser-web.
• Object storage: Tigris S3-compatible bucket for uploads and artifacts.

Communication in production:

• Browser ↔ poser-web for auth, uploads, results, and the embed widget (static assets are served by poser-web).
• poser-web ↔ poser-analysis via Flycast (ANALYSIS_SERVICE_URL).
• poser-analysis ↔ poser-web internal endpoints for progress and artifacts.
• poser-web ↔ poser-db for persistence.
• Both services ↔ Tigris for file storage.

CI workflow

• Feature branches: Open a PR to main to trigger lint + test jobs.
• Main branch: On merge, CI runs again and deploys to Fly.io.
• Checks: Ruff lint/format, analysis compile checks, Rails specs, frontend tests.
• Deploy: GitHub Actions uses flyctl deploy with fly.web.toml and fly.analysis.toml.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• YOLO26 by Ultralytics
• MediaPipe by Google
• BoT-SORT tracking algorithm
• Open source computer vision community