Skip to content

feat: Add SONIC locomotion controller (CPU inference via onnxruntime) #86

Description

@MiaoDX

Context

Follow-up from #82. The SONIC controller from NVlabs has been investigated and deferred due to architectural complexity.

Research Findings

SONIC ONNX models are available at nvidia/GEAR-SONIC on HuggingFace. The C++ inference code in NVlabs/GR00T-WholeBodyControl confirms CPU inference is feasibleOrtSession uses CPUExecutionProvider with no GPU/CUDA references.

However, the architecture is significantly more complex than GR00T/Holosoma:

Aspect GR00T / Holosoma SONIC
ONNX models 1-2 single models 3 models (encoder + decoder + planner)
Input tensors 1 (obs vector) 6-11 (context qpos, velocity, mode, direction, seed, etc.)
Observation Single vector (86-100D) 4-frame context window (4 × 36) + command inputs
Locomotion modes Balance/Walk or single policy IDLE, SLOW_WALK, WALK, RUN, BOXING

Proposed Implementation

Add SonicLocomotionController to src/roboharness/controllers/locomotion.py:

  1. Planner-only mode: Use planner_sonic.onnx for locomotion (keyboard/gamepad-style velocity commands → joint targets). This is the simplest entry point and most analogous to GR00T/Holosoma.
  2. Full pipeline mode (stretch): Wire encoder + decoder for motion tracking / VR teleoperation inputs.

Key details for planner (LocalMotionPlannerONNX):

  • Input (v0, 6 tensors): context_mujoco_qpos [4×36], target_vel [1], mode [1], movement_direction [3], facing_direction [3], random_seed [1]
  • Output: Predicted qpos frames [N×36] + frame count
  • Modes: 0=IDLE, 1=SLOW_WALK, 2=WALK, 3=RUN, 4=BOXING

References

Parent issue: #82

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions