Skip to content

Mocap Slice G — Model hosting, docs, packaging, CI, telemetry #876

Description

@fernandotonon

Part of epic #869. Final hardening slice — hosting, packaging, docs, CI, release readiness.

Model hosting

  • scripts/upload-mocap-models.sh (clone upload-triposr-models.sh): push to HF fernandotonon/QtMeshEditor-models under mocap/face/ (detector + landmarks + blendshapes ONNX) and — if Slice A's license verdict passed — mocap/body/ (SAM 3D Body ONNX + MHR skeleton JSON) and mocap/pose/ (pose landmarker ONNX). Include the license files next to the weights as the licenses require (Apache-2.0 NOTICE for MediaPipe; SAM License text if applicable).
  • Verify the first-run download path end-to-end from a clean AppData on macOS + Linux; verify QTMESH_MOCAP_NO_DOWNLOAD offline guard and the "not yet hosted" graceful message (the TripoSG precedent).

Docs

  • docs/MOCAP.md: user-facing guide — supported inputs, the ARKit-52 morph-target requirement (with pointers to Ready Player Me / CC as sources of compatible heads and to the Anim: Slice B — Mesh / vertex animation (Alembic + Ogre VAT_POSE clips) #519 authoring UX for making targets manually), mapping-override JSON format, body-capture rig requirements (humanoid, ≥½ canonical roles), known limitations (single person, foot slide, camera-relative head pose), tuning (One-Euro params, fps).
  • THIRD_PARTY_AI_MODELS.md: MediaPipe (Apache-2.0), SAM 3D Body verdict from Slice A, MHR (Apache-2.0), rejected-alternatives table (FLAME-based face models, SMPL-based body models, OpenPose, FreeMoCap) — the house decision-record format.
  • CLAUDE.md: add the qtmesh mocap examples to the CLI block + a Mocap architecture section (feature folder, predictors, ENABLE_MOCAP, model env vars) following the existing feature-section style.
  • README.md feature list + action.yml command description (new CLI subcommand → the action doc rule in CLAUDE.md).

Packaging / CI

  • Release builds: flip ENABLE_MOCAP=ON on macOS + Linux release lanes once green; Windows/MinGW per the Slice B verification (if Qt Multimedia MinGW works, enable; else document OFF like ONNX/MinGW).
  • Debian control: add Qt Multimedia runtime deps; verify the Docker image still builds (headless container: camera absent is fine — file-based qtmesh mocap must work under Xvfb with the FFmpeg backend; add a scan-assets-docker-style smoke job step running qtmesh mocap --frames-dir on checked-in frames).
  • macdeployqt bundles the Multimedia plugin + FFmpeg libs; smoke-test the notarized bundle's camera permission prompt.
  • Linux coverage lane runs the full mocap test suite (models env-gated where needed).

Telemetry / gamification

  • Sentry breadcrumb audit: ai.assist.mocap_face, ai.assist.mocap_body (+algo), ai.assist.mocap_live, ui.action on all panel controls, file.import/file.export on the CLI paths — verify against the categories table in CLAUDE.md.
  • Gamification: add the mocap feature cluster key to Gamification::featureCatalog() and coordinate the matching DISCOVERY_FEATURES entry in qtmesh-cloud (the contract note in CLAUDE.md — keys must match); noteFeature/noteOperation call-site audit across GUI/CLI/MCP.
  • MCP tool → cluster mapping in MCPServer::callTool; CLI subcommand → cluster in CLIPipeline::run.

Final QA matrix

  • macOS: live capture (camera permission prompt → grant → preview → record → undo → export), file capture, all fallback messages.
  • Linux (incl. Docker/Xvfb): file capture face + body, --no-model fallback, JSON reports schema-stable.
  • Windows: whichever tier Slice B landed (full / file-only / clear "rebuild" message) behaves as documented.
  • Non-MOCAP build: every surface (CLI, MCP schema/gating, GUI section) shows the documented message, zero crashes.
  • Fresh-machine first-run: model downloads with progress, then works offline thereafter.

Acceptance criteria

  • Models hosted + first-run download verified from clean state.
  • All docs above landed; CLAUDE.md CLI block updated; action.yml updated.
  • Release lanes green with ENABLE_MOCAP=ON (per-platform tier documented).
  • Telemetry + gamification audit complete (cloud-side key coordinated).

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-assistLocal-AI-assisted 3D workflows (epic prefix: AI:)animationAnimation systems: skeletal, morph, pose, VAT, alembic, proceduralenhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions