Surgical Robotics Challenge

For more information regarding the challenge, please visit Surgical Robotics Challenge 2021-2022 or Surgical Robotics Challenge 2023-2024.

Discussions Forum

Please checkout the Discussions Tab for asking questions, posting suggestions, connecting with the community, and for keeping up to date with the challenge.

1. Install AMBF and ROS Prerequisites

Ensure that the correct version of ROS is installed and sourced on your system. For streaming the image and depth data out of AMBF, please also install the following ROS packages

• cv_bridge
• image_transport

apt-get install ros-<version>-cv-bridge ros-<version>-image-transport

Then, clone, build, and source AMBF's ambf-2.0 branch.

https://github.com/WPI-AIM/ambf/tree/ambf-2.0

First time cloning:

git clone https://github.com/WPI-AIM/ambf.git
cd ambf
git checkout -b ambf-2.0 origin/ambf-2.0

In case there are updates to AMBF, you can simply update your local copy by:

git pull

Don't forget to build the repo using the instructions on AMBF's Readme. The ambf_simulator binary resides in <path>/ambf/bin/lin-x86_64, the <path> is where you cloned AMBF, In the AMBF instructions the <path> is the home folder ~, so the correct path is ~/ambf/bin/lin-x86_64. If you haven't done so, please create an alias for the ambf_simulator in your .bashrc file. This way you do not need to be in the <path>/ambf/bin/lin-x86_64 folder to run ambf_simulator. The alias can be created by typing (make sure that the path ~/ambf/bin/lin-x86_64 is the correct path in the command below):

echo "alias ambf_simulator=~/ambf/bin/lin-x86_64/ambf_simulator" >> ~/.bashrc

Restart any running terminals for the alias to be available in them.

2. Clone this repo to your local machine OR use a Dockerfile

Option 1: (Clone repo to your local machine)

Please refer to README in the scripts folder for instructions on installing the Python package for system-wide access.

Option 2: (Use Dockerfile)

You can alternatively use Dockerfiles to create Docker images by following the instructions here:

https://github.com/surgical-robotics-ai/docker_surgical_robotics_challenge

3. Running the simulation

For convenience, we have provided several .sh scripts which can be used directly to launch the surgical scenes. For example, with roscore already running, you can run the run_env_LND_420006.sh script in your terminal as:

 ./run_env_LND_420006.sh

and you should see the following scene

This script assumes that AMBF is located in your home ~ folder. If it is not, please edit the scripts accordingly.

3a. The launch file:

To understand the launch file, visit the following link:

https://github.com/WPI-AIM/ambf/wiki/Selecting-Robots

3b. Simulated Cameras

The simulated camera(s) is defined in the World file (world_stereo.yaml) which is set in the launch.yaml file. To enable the camera(s) to publish the scene image or depth data, follow the instructions on this page:

https://github.com/WPI-AIM/ambf/wiki/Camera-feed-and-depth-camera

3c. Camera Coordinate frames

To better understand the different camera coordinate frames and the difference between the AMBF and the Opencv camera convention, please refer to camera_convention.md

3c. Resetting the Simulation

You can press CTRL+R to reset the rigid bodies in simulation, and CTRL+V to reset the camera pose if you changed it with the mouse.

3d. Launch Arguments:

To manually control what objects are spawing in the scene, please review the .sh scripts in this folder. For a full list of arguments to provide to AMBF, please refer to this link:

https://github.com/WPI-AIM/ambf/wiki/Command-Line-Arguments

4. Interacting with Simulated Robots using Python Scripts:

Please take a look at the scripts in the scripts folder:

5. Controlling via Input Devices

The code in the scripts folder allows the dVRK MTMs or Geomagic Touch / Phantom Omni to control the simulated PSMs.

With the simulation already running, run the dvrk-ros application for the dVRK MTMs or the ROS application for the Geomagic Touch/Phantom Omni. You can find the relevant code for them here:

a. https://github.com/jhu-dvrk/dvrk-ros (dvrk-ros)

b. https://github.com/WPI-AIM/ros_geomagic (geomagic_touch/phantom_omni)

Then run one of the corresponding Python scripts:

a. scripts/surgical_robotics_challenge/teleoperation/mtm_multi_psm_control.py (For MTMs)

b. scripts/surgical_robotics_challenge/geomagic_multi_psm_control.py (For Geomagic Touch/Phantom Omni)

Refer to the README in the scripts folder for further information

6. Citation

If you find this work useful, please cite it as:

@article{munawar2022open,
  title={Open Simulation Environment for Learning and Practice of Robot-Assisted Surgical Suturing},
  author={Munawar, Adnan and Wu, Jie Ying and Fischer, Gregory S and Taylor, Russell H and Kazanzides, Peter},
  journal={IEEE Robotics and Automation Letters},
  volume={7},
  number={2},
  pages={3843--3850},
  year={2022},
  publisher={IEEE}
}