Context
Follow-up from #82. The SONIC controller from NVlabs has been investigated and deferred due to architectural complexity.
Research Findings
SONIC ONNX models are available at nvidia/GEAR-SONIC on HuggingFace. The C++ inference code in NVlabs/GR00T-WholeBodyControl confirms CPU inference is feasible — OrtSession uses CPUExecutionProvider with no GPU/CUDA references.
However, the architecture is significantly more complex than GR00T/Holosoma:
| Aspect |
GR00T / Holosoma |
SONIC |
| ONNX models |
1-2 single models |
3 models (encoder + decoder + planner) |
| Input tensors |
1 (obs vector) |
6-11 (context qpos, velocity, mode, direction, seed, etc.) |
| Observation |
Single vector (86-100D) |
4-frame context window (4 × 36) + command inputs |
| Locomotion modes |
Balance/Walk or single policy |
IDLE, SLOW_WALK, WALK, RUN, BOXING |
Proposed Implementation
Add SonicLocomotionController to src/roboharness/controllers/locomotion.py:
- Planner-only mode: Use
planner_sonic.onnx for locomotion (keyboard/gamepad-style velocity commands → joint targets). This is the simplest entry point and most analogous to GR00T/Holosoma.
- Full pipeline mode (stretch): Wire encoder + decoder for motion tracking / VR teleoperation inputs.
Key details for planner (LocalMotionPlannerONNX):
- Input (v0, 6 tensors):
context_mujoco_qpos [4×36], target_vel [1], mode [1], movement_direction [3], facing_direction [3], random_seed [1]
- Output: Predicted qpos frames [N×36] + frame count
- Modes: 0=IDLE, 1=SLOW_WALK, 2=WALK, 3=RUN, 4=BOXING
References
Parent issue: #82
Context
Follow-up from #82. The SONIC controller from NVlabs has been investigated and deferred due to architectural complexity.
Research Findings
SONIC ONNX models are available at
nvidia/GEAR-SONICon HuggingFace. The C++ inference code in NVlabs/GR00T-WholeBodyControl confirms CPU inference is feasible —OrtSessionusesCPUExecutionProviderwith no GPU/CUDA references.However, the architecture is significantly more complex than GR00T/Holosoma:
Proposed Implementation
Add
SonicLocomotionControllertosrc/roboharness/controllers/locomotion.py:planner_sonic.onnxfor locomotion (keyboard/gamepad-style velocity commands → joint targets). This is the simplest entry point and most analogous to GR00T/Holosoma.Key details for planner (
LocalMotionPlannerONNX):context_mujoco_qpos[4×36],target_vel[1],mode[1],movement_direction[3],facing_direction[3],random_seed[1]References
Parent issue: #82