Sim2E

We introduce our Mujoco based, Interbotix vx300s digital twin with six degrees of freedom (DOF) robotic arm, mounted with a virtual event camera. The robotic arm 3D model is in a configurable virtual lab, rendered in the Unity3D game engine, and the events are generated using the Esim event camera simulator. The system can generate, and record simulated neuromorphic vision data while the robot arm is in motion.

This project utilizes Esim to generate events from a sequence of images. You can find a pdf of the paper here. If you use any of this code, please cite this publication.

@Article{Rebecq18corl,
  author        = {Henri Rebecq and Daniel Gehrig and Davide Scaramuzza},
  title         = {{ESIM}: an Open Event Camera Simulator},
  journal       = {Conf. on Robotics Learning (CoRL)},
  year          = 2018,
  month         = oct
}

Installation

Requirements

• Ubuntu 18.04 or newer
• Nvidia GPU with CUDA support
• Nvidia Cudnn
• Anaconda

step 0: Create a conda environment

conda create -n sim2e python=3.9
conda activate sim2e

step 1: Install python binding of Esim with GPU support

IMPORTANT: Make sure to stay on the sim2e conda environment and don't switch to the environment in the Esim installation guide.
Instructions can be found in this github page.

step 2: Install Mujoco

Install mujoco and mujoco-py. Instructions can be found in this video here . You can also follow the instructions in this document instead.

step 3: Clone the repository and install dependencies

git clone https://github.com/Turgibot/Sim2E.git
cd Sim2E
pip install -r requirements.txt

Usage

Run the following command to start the simulation

python main.py