feat(devices): add Meta Quest 3 body tracking via OSC protocol

[brukg]

Add support for Meta Quest 3 body tracking through OSC protocol for full-body humanoid robot teleoperation. This enables users to use
Meta Quest 3's inside-out body tracking cameras with ALVR and SteamVR to control humanoid robots in Isaac Lab.

Changes:

• Add BodyOscReceiver class for receiving body tracking data via OSC (VRChat protocol compatible)
• Add BODY tracking target to DeviceBase and BODY_TRACKING requirement to RetargeterBase
• Add configurable OSC port via OpenXRDeviceCfg.body_osc_port parameter
• Add keyboard toggle controls (S key for teleoperation, R key for reset) to Se2Keyboard
• Improve GR1T2Retargeter with pre-computed joint index mappings for better performance
• Add comprehensive how-to guide for Meta Quest 3 setup with ALVR and SteamVR

Dependencies:

• python-osc package (installed via ./isaaclab.sh -p -m pip install python-osc)

Type of change

• New feature (non-breaking change which adds functionality)
• Documentation update

Checklist

• I have read and understood the contribution
  guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Greptile Summary

• Adds Meta Quest 3 body tracking support via OSC protocol for full-body humanoid robot teleoperation, including BodyOscReceiver class for receiving VRChat-compatible body tracking data
• Extends device infrastructure with new BODY tracking target and BODY_TRACKING requirement to support body-aware retargeters alongside existing hand/head tracking capabilities
• Provides comprehensive documentation and setup guide for configuring Meta Quest 3 with ALVR and SteamVR for real-time body tracking teleoperation

Important Files Changed

Filename	Overview
source/isaaclab/isaaclab/devices/openxr/osc_receiver.py	New OSC receiver class for body tracking data with threading and coordinate transformation
source/isaaclab/isaaclab/devices/openxr/openxr_device.py	Integrates OSC body tracking into OpenXR device with lazy initialization and quaternion format conversion
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/fourier/gr1t2_retargeter.py	Enhanced retargeter with body tracking support and performance optimizations via pre-computed joint indices
scripts/environments/teleoperation/teleop_se3_agent.py	Adds keyboard controls with potential callback conflicts between main teleop interface and new Se2Keyboard instance

Confidence score: 3/5

• This PR implements a comprehensive new feature with proper documentation and follows established patterns, but has some areas requiring careful review
• Score reflects concerns about potential callback conflicts in keyboard handling, dependency management, and the complex integration of external OSC protocol communication
• Pay close attention to threading safety in OSC receiver implementation and potential conflicts in keyboard callback handling across multiple device instances

Sequence Diagram

sequenceDiagram
    participant User as User
    participant Quest as Meta Quest 3
    participant ALVR as ALVR
    participant OSCReceiver as BodyOscReceiver
    participant OpenXRDevice as OpenXRDevice
    participant GR1T2Retargeter as GR1T2Retargeter
    participant Robot as GR1T2 Robot

    User->>Quest: "Performs body movements"
    Quest->>ALVR: "Captures body tracking data"
    ALVR->>OSCReceiver: "Sends OSC messages on port 9000"
    OSCReceiver->>OSCReceiver: "Updates body joint positions"
    OSCReceiver->>OSCReceiver: "Computes heuristic orientations"
    
    User->>Quest: "Performs hand gestures"
    Quest->>OpenXRDevice: "Provides hand tracking via OpenXR"
    
    OpenXRDevice->>OpenXRDevice: "Calculates joint poses for hands"
    OpenXRDevice->>OSCReceiver: "Requests body tracker poses"
    OSCReceiver-->>OpenXRDevice: "Returns body tracking data"
    
    OpenXRDevice->>GR1T2Retargeter: "Sends raw tracking data"
    GR1T2Retargeter->>GR1T2Retargeter: "Retargets to robot commands"
    GR1T2Retargeter-->>OpenXRDevice: "Returns robot joint commands"
    
    OpenXRDevice-->>Robot: "Applies retargeted commands"
    Robot->>Robot: "Executes movements"

Loading

[greptile-apps]

_{15 files reviewed, 4 comments}

_{Edit Code Review Agent Settings | Greptile}