|
| 1 | +# Detailed descriptions on the python teleoperation examples |
| 2 | + |
| 3 | +## XR client |
| 4 | +- https://github.com/XR-Robotics/XRoboToolkit-Teleop-Sample-Python/blob/main/xrobotoolkit_teleop/utils/xr_client.py |
| 5 | +- connects to XR device using the python binding of [xrobotoolkit-sdk](https://github.com/XR-Robotics/XRoboToolkit-PC-Service-Pybind) |
| 6 | + |
| 7 | +## Placo |
| 8 | +- [PlaCo](https://placo.readthedocs.io/en/latest/) is an implementation for whole-body inverse kinematics and dynamics based on Quadratic Programming solver. |
| 9 | +- Tasks: following are the main placo tasks used in this repo |
| 10 | + - Frame Task |
| 11 | + - Constraints position and orientation of a frame, used to define end-effector tracking goals for IK |
| 12 | + - Manipulability Task |
| 13 | + - Makes robot motion stable when commanded position is close to singularity or outside of workspace |
| 14 | + - Based on this paper: https://arxiv.org/abs/2002.11901 |
| 15 | + - Kinetic Energy Regularization |
| 16 | + - Minimizes the kinetic energy of the system |
| 17 | + - Joint Task |
| 18 | + - Regularizes the joint state to be close to the default state |
| 19 | + |
| 20 | +## Mujoco simulation |
| 21 | +- Robot definition files: both `.xml` and `.urdf` files are required and they should be consistent with each other (same link names and joint names). The `.xml` file is for mujoco simulation, and the `.urdf` is for placo. Optionally, there should be 1 additional free floating body per end effector defined in the `.xml` file for visualization of commanded teleop targets in mujoco. |
| 22 | +- Teleoperation task is defined by a config dict |
| 23 | + - link_name: the name of end effector link as defined in mujoco .xml & .urdf files |
| 24 | + - pose_source: name of the source of pose to be used by Pico client |
| 25 | + - control_trigger: the key to define whether an arm control is active |
| 26 | + - vis_target: name of the body for teleop target visualization |
| 27 | + ```python |
| 28 | + config = { |
| 29 | + "right_hand": { |
| 30 | + "link_name": "right_tool0", |
| 31 | + "pose_source": "right_controller", |
| 32 | + "control_trigger": "right_grip", |
| 33 | + "gripper_trigger": "right_trigger", |
| 34 | + "vis_target": "right_target", # optional, only used in mujoco |
| 35 | + }, |
| 36 | + "left_hand": { |
| 37 | + "link_name": "left_tool0", |
| 38 | + "pose_source": "left_controller", |
| 39 | + "control_trigger": "left_grip", |
| 40 | + "gripper_trigger": "left_trigger", |
| 41 | + "vis_target": "left_target", # optional, only used in mujoco |
| 42 | + }, |
| 43 | + } |
| 44 | + ``` |
| 45 | + |
| 46 | +- Run mujoco demo for dual UR5e with the following script |
| 47 | + ```bash |
| 48 | + python scripts/simulation/teleop_dual_ur5e_mujoco.py |
| 49 | + ``` |
| 50 | + |
| 51 | +- Controlling parallel gripper in mujoco simulation |
| 52 | + - Users can add an optional gripper configuration in the end effector config dict |
| 53 | + - joint_name: the actuated mujoco joint within the gripper |
| 54 | + - gripper_trigger: name of the key mapped to this gripper from the controller |
| 55 | + - open_pos: the value of the actuated joint when fully opened |
| 56 | + - close_pos: the value of the actuated joint when fully closed |
| 57 | + ```python |
| 58 | + config = { |
| 59 | + "right_hand": { |
| 60 | + # other configs, |
| 61 | + "gripper_config": { |
| 62 | + "joint_name": "right_gripper_finger_joint1", |
| 63 | + "gripper_trigger": "right_trigger", |
| 64 | + "open_pos": 0.05, |
| 65 | + "close_pos": 0.0, |
| 66 | + }, |
| 67 | + }, |
| 68 | + "left_hand": { |
| 69 | + # other configs, |
| 70 | + "gripper_config": { |
| 71 | + "joint_name": "left_gripper_finger_joint1", |
| 72 | + "gripper_trigger": "left_trigger", |
| 73 | + "open_pos": 0.05, |
| 74 | + "close_pos": 0.0, |
| 75 | + }, |
| 76 | + }, |
| 77 | + } |
| 78 | + ``` |
| 79 | + - Note that the parallel gripper might contain multiple joints in the `.xml` file, but only 1 of the joints should be actuated, the others should be controlled by additional equality constraints in the xml. The `.urdf` file supplied to Placo does not have to contain the gripper dof. |
| 80 | + ```xml |
| 81 | + <equality> |
| 82 | + <joint name="right_gripper_constraint" joint1="right_gripper_finger_joint1" joint2="right_gripper_finger_joint2" polycoef="0 -1 0 0 0" /> |
| 83 | + <joint name="left_gripper_constraint" joint1="left_gripper_finger_joint1" joint2="left_gripper_finger_joint2" polycoef="0 -1 0 0 0" /> |
| 84 | + </equality> |
| 85 | + ``` |
| 86 | +- Example of mujoco teleoperation with gripper control using dual A1X arm |
| 87 | + ```bash |
| 88 | + python scripts/simulation/teleop_dual_a1x_mujoco.py |
| 89 | + ``` |
| 90 | + |
| 91 | +## Hardware teleoperation: |
| 92 | +- Robot definition files: only `.urdf` file is required |
| 93 | +- Config |
| 94 | + - link_name: the name of end effector link as defined in mujoco .xml & .urdf files |
| 95 | + - pose_source: name of the source of pose to be used by Pico client |
| 96 | + - control_trigger: the key to define whether an arm control is active |
| 97 | + - gripper_trigger: name of the key mapped to this gripper from the controller |
| 98 | + ```python |
| 99 | + DEFAULT_END_EFFECTOR_CONFIG = { |
| 100 | + "left_arm": { |
| 101 | + "link_name": "left_tool0", |
| 102 | + "pose_source": "left_controller", |
| 103 | + "control_trigger": "left_grip", |
| 104 | + "gripper_trigger": "left_trigger", |
| 105 | + }, |
| 106 | + "right_arm": { |
| 107 | + "link_name": "right_tool0", |
| 108 | + "pose_source": "right_controller", |
| 109 | + "control_trigger": "right_grip", |
| 110 | + "gripper_trigger": "right_trigger", |
| 111 | + }, |
| 112 | + } |
| 113 | + ``` |
| 114 | +- Example code is provided for a lab setup with dual ur5e manipulators with robotiq 2f85 grippers, as well as a 2 DOF head controlled by 2 dynamixel motors. |
| 115 | + ```bash |
| 116 | + python scripts/hardware/teleop_dual_ur5e_hardware.py |
| 117 | + ``` |
0 commit comments