Skip to content

[codex] improve client thermal efficiency and trace reporting#135

Draft
human-bee wants to merge 2 commits intomainfrom
codex/thermal-efficiency-livekit-crowdpulse
Draft

[codex] improve client thermal efficiency and trace reporting#135
human-bee wants to merge 2 commits intomainfrom
codex/thermal-efficiency-livekit-crowdpulse

Conversation

@human-bee
Copy link
Owner

Summary

This PR reduces browser-side thermal load and duplicate client work in production canvas sessions, and strengthens observability artifacts for showcase validation.

User Impact / Problem

On https://present.best/canvas, renderer CPU/GPU usage was significantly higher than expected for server-centric canvas operations. In production traces, we also saw noisy proof artifacts that were hard to reconcile with UI behavior.

Root Cause

  1. Canvas LiveKit autopilot could auto-connect too eagerly and publish local media / auto-request agents even for passive parity sessions.
  2. Each CrowdPulseWidget created its own camera + MediaPipe hand tracker loop in the browser, with RAF-driven wakeups.
  3. Shared sensor startup lacked cancellation guards, which could leave camera/model work running if subscribers disappeared mid-start.
  4. Showcase traces lacked a first-class HTML artifact and had partial metric/report mismatches (missingTraceOnTrackedTasks gate vs displayed chips).

What Changed

LiveKit / Canvas parity

  • Added publishLocalMedia and autoRequestAgent options in LiveKit hook stack:
    • src/components/ui/livekit/hooks/useLivekitConnection.ts
    • src/components/ui/livekit/hooks/useLkAgentRequest.ts
    • src/components/ui/livekit/hooks/useLkToken.ts
    • src/components/ui/livekit/hooks/utils/lk-connect.ts
  • CanvasParityAutopilot now:
    • uses explicit backoff reconnect,
    • disables local media publication,
    • disables auto agent requests,
    • avoids implicit demo-mode autoconnect churn.

Crowd Pulse sensor runtime

  • Introduced shared sensor manager:
    • src/components/ui/productivity/crowd-pulse-sensor.ts
  • Refactored widget to consume shared manager:
    • src/components/ui/productivity/crowd-pulse-widget.tsx
  • Efficiency improvements:
    • one camera stream + one hand landmarker per tab,
    • timer-based sampling (not RAF hot loop),
    • lower capture resolution (480x270),
    • fail-stop on detector errors,
    • startup cancellation token to prevent orphan camera/model loops.

Showcase / observability artifacts

  • Added reusable trace HTML renderer:
    • scripts/observability/render-agent-trace-html.mjs
  • Wired both scripts to emit agent-trace.html and persist agentTraceHtml path:
    • scripts/observability/playwright-voice-showcase-loop.mjs
    • scripts/observability/replay-trace-canvas.mjs
  • Improved proof logic/reporting alignment:
    • include agentReady in result,
    • use tracked-task trace integrity metric for gate,
    • accept executing as dispatch evidence fallback,
    • surface executing and missingTraceOnTrackedTasks in HTML chips.

Backward Compatibility

  • No DB migrations.
  • No external API contract changes.
  • Behavior change is client-runtime only: parity autopilot no longer implicitly autoconnects in demo mode (intentional thermal safety).

Validation

  • npm run typecheck
  • npm test -- src/components/ui/productivity/__tests__/crowd-pulse-widget.test.tsx src/components/ui/productivity/__tests__/crowd-pulse-hand-utils.test.ts src/lib/agents/subagents/canvas-steward.test.ts src/lib/agents/conductor/__tests__/router-execute-task.test.ts
  • node --check scripts/observability/playwright-voice-showcase-loop.mjs
  • node --check scripts/observability/replay-trace-canvas.mjs
  • node --check scripts/observability/render-agent-trace-html.mjs
  • npm run lint (passes with existing repo-wide warnings; no new lint blocker introduced)

Reviewer Lanes

Reviewer A (correctness/perf/regression)

Findings:

  • High: shared sensor startup race could leak camera/CPU when subscribers drop mid-init.
  • Low: report chip mismatch for tracked-trace metric.

Resolution:

  • Added startup token cancellation checks across async startup steps.
  • Added tracked metric chips (missingTraceOnTrackedTasks, executing) to trace HTML.

Reviewer B (maintainability/architecture)

Findings:

  • Confirmed same high/low issues independently.
  • Flagged demo-mode autoconnect drift as medium risk.

Resolution:

  • Kept demo autoconnect change intentionally (thermal guard), documented in PR and code comment.

Evidence Artifacts

Local artifacts generated during verification:

  • Stable showcase (UI matched artifact intent):
    • test-results/observability-showcase/showcase-1772098808024-8bafb2b7/de05d54b1fa6bca4cda3d34f0b78f3ab.webm
    • test-results/observability-showcase/showcase-1772098808024-8bafb2b7/agent-trace.html
    • test-results/observability-showcase/showcase-1772098808024-8bafb2b7/result.json
  • Additional flaky production captures (useful forensic evidence for remaining _zod fairy retries):
    • test-results/observability-showcase/showcase-1772098936151-328aed0d/result.json
    • test-results/observability-showcase/showcase-1772099129654-da496d40/result.json

Remaining Risks / Follow-ups

  • Production still shows intermittent fairy.intent _zod failures in some runs, causing retries and partial shape fulfillment in those sessions. This PR improves thermal efficiency and observability, but does not fully remediate that upstream fairy pipeline instability.

@vercel
Copy link

vercel bot commented Feb 26, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
present Error Error Feb 26, 2026 8:48pm

Request Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant