ROS2 workspace and example nodes

[tsampazk]

This PR contains a new ROS2 (Foxy-Fitzroy version) workspace located in the projects directory and, for now, it serves to test and finalize the structure, naming, etc., gather and finalize all ROS2 nodes in a unified PR. Right now there are no docstrings (docstrings added), documentation or READMEs. This description will get updated for any additions.

Contents:

1. opendr_perception python package
   • Contains a pose estimation node
   • Contains fall detection node
   • Contains object detection 2d centernet/detr/ssd/yolov3 nodes
   • Contains face detection retinaface node
   • Contains face recognition node
   • Contains semantic segmentation bisenet node
   • Contains a subscriber tester node (tester), that subscribes to the messages published by the pose estimation node for
     testing Testing can be performed as described in steps 9 and 10 of Building and Running below.
2. opendr_ros2_bridge python package
   • Contains bridge.py which includes a class with methods to convert images, poses, etc. from and to ROS2 messages
   • This uses cv_bridge which is included in the vision_opencv package
3. opendr_ros2_messages CMake package
   • Contains custom OpenDR messages, right now only OpenDRPose2D and OpenDRPose2DKeypoint
   • Note that this package is a CMake package due to limitations in ROS2: custom messages cannot be generated in python packages

The logic behind the structuring of the packages and nodes is similar to OpenDR's ROS1 packages/nodes.

Below you can find instructions to install, build and run the nodes for testing. Note that i did everything on a system with ROS1 already installed.

I faced many issues along the way that might reappear in a fresh install of ROS2, etc., so if any problems/errors occur following the instructions please get in touch with me, to possibly save you some time.

Installation

• To install ROS2 i followed this tutorial (section 2), which installs the 'foxy' release of ROS2. (Note that on '(7) configure environment variables', you need to replace dashing with foxy)
• Edit: At this point you might need to run sudo apt-get install ros-foxy-vision-msgs as discussed below
• Install colcon, basically just sudo apt install python3-colcon-common-extensions
• Install ros2 usb cam to test with local webcam. In my case i use ros2 run usb_cam usb_cam_node_exe to run it after installation, which seems to work fine

Building and Running

1. Navigate to your OpenDR installation and activate it as usual
2. Navigate to workspace root, opendr_ws_2 directory
3. Install cv_bridge via the instructions in its README, excluding the last step(build). There seems to be no need to build it, as it will get built along with the rest of the packages later.
4. Navigate to the workspace root (opendr_ws_2) as the previous step leaves you inside vision_opencv dir
5. Run colcon build
6. Run . install/setup.bash
7. Run ros2 run opendr_perception pose_estimation to start the pose estimation node (or any other existing node)
8. In a new terminal run ros2 run usb_cam usb_cam_node_exe to grab images from a webcam
9. In a new terminal run ros2 run rqt_image_view rqt_image_view and select the corresponding topic to view the image result
10. In a new terminal run ros2 topic echo opendr/poses to view the pose message. Note that it is not really human readable in that form, it should be read in another node and converted into an OpenDR pose object to have access to human-friendly print methods.

* If you are using conda, check out Illia's comment down below. Thanks @iliiliiliili !

To be added

ROS2 nodes to be added according to what ROS1 nodes exist already:

---

Perception package:

• Object detection 2D detr (update from original author) (Ros2 detr #296)
• Video activity recognition (Ros2 har #323)
• RGBD hand gesture recognition (ROS2 RGBD hand gestures recognition #341)
• Panoptic segmentation EfficientPS (Add ROS2 node for EfficientPS #270)
• Heart anomaly detection(ROS2 for heart anomaly detection #337)
• Speech command recognition ( ROS2 speech command recognition #340)
• Audiovisual emotion recognition (ROS2 for audiovisual emotion recognition node #342)
• Skeleton based action recognition (Skeleton_based_action_recognition ROS2 node #344)
• Landmark-based facial expression recognition (landmark_based_facial_expression_recognition ROS2 node #345)
• Image-based facial emotion estimation (new tool Fer va estimation #264, Image-based facial emotion estimation ROS2 node #346)
• Object detection 2D gem (Ros2 gem #295)
• Object detection 2D YOLOv5 (added in Object detection 2d yolov5 #360, I will directly add the ROS2 node on ros2 branch)
• Object detection 2D Nanodet (added in Nanodet #278, I will directly add the ROS2 node on ros2 branch)
• Object tracking 2D SiamRPN (added in siamrpn: Single Object tracking 2d #367, I will directly add the WIP ROS2 node on ros2 branch)
• High resolution pose estimation (added in High Resolution Pose Estimation new tool #356, I will directly add the ROS2 node on ros2 branch)
• Image dataset (Ros2 nodes for 3D Detection and 2D/3D tracking #319)
• Point cloud dataset (Ros2 nodes for 3D Detection and 2D/3D tracking #319)
• Object detection 3D voxel (Ros2 nodes for 3D Detection and 2D/3D tracking #319)
• Object tracking 2D deep sort (Ros2 nodes for 3D Detection and 2D/3D tracking #319)
• Object tracking 2D fair mot (Ros2 nodes for 3D Detection and 2D/3D tracking #319)
• Object tracking 3D ab3dmot (Ros2 nodes for 3D Detection and 2D/3D tracking #319)

---

Data generation package:

• Synthetic facial recognition (Ros2 synthetic facial generation #288)

---

Simulation package:

• Human model generation client/service (ROS2 implementation for human_model_generation module #291)

---

Planning package:

• End to end planner (this is new for ROS1 too) (Ros nodes e2e planner #286, new PR will be opened for ROS2 ROS2 node for e2e planner and environment upgrade to Webots R2022b #358)

---

Edit1: Updated the last steps of the instructions as well as the contents list as per the latest changes.
Edit2: Added information in contents list about the new opendr_ros2_messages. added TODO list for remaining nodes

[tsampazk]

Hey @ad-daniel, i have started working on implementing ROS2 nodes for object detection and semantic segmentation.

Should i keep adding the new stuff on the ros2 branch and consequently on this PR, or do you think we should handle it differently?

[ad-daniel]

Ideally it would be better to merge a minimal version (with some changes like adding vision_opencv as a submodule, re-organize some stuff, ...) so that other partners can do their own branches from develop and begin the development in parallel. In practice however since develop is currently frozen in preparation for the release, we should probably wait for that (either way it will occur this week).

For you and others that wish to begin the ros2 development I think the best approach at the moment might be to create a branch from THIS branch. Once the release has occurred, we can do the necessary changes in this PR and merge it right after.
@passalis @stefaniapedrazzi unless you guys see a better way

[tsampazk]

For you and others that wish to begin the ros2 development I think the best approach at the moment might be to create a branch from THIS branch. Once the release has occurred, we can do the necessary changes in this PR and merge it right after.

Alright Daniel, thanks! I also thought that might be a good potential way of doing things.

[vniclas]

@tsampazk thanks for your detailed instructions.
I tried following your steps but am running into the following issue ModuleNotFoundError: No module named 'vision_msgs'. Note that I am using a python virtual environment as created by the ./bin/install.sh script. Have you experienced something similar?

[tsampazk]

@tsampazk thanks for your detailed instructions. I tried following your steps but am running into the following issue ModuleNotFoundError: No module named 'vision_msgs'. Note that I am using a python virtual environment as created by the ./bin/install.sh script. Have you experienced something similar?

Hey @vniclas ! Could you run this rosdep install -i --from-path src --rosdistro foxy -y on the root workspace directory and try again?

[vniclas]

The tests as described in the vision_opencv/cv_bridge readme pass. Sadly, I cannot run your proposed command due to the same error cannot locate rosdep definition for [vision_msgs].
I tried rebuilding the entire OpenDR environment from scratch using this branch but it doesn't succeed anymore. However, I'm not sure that those errors are related. I'll open a bug ticket tomorrow. In a nutshell, I can't build the C++ package face_recognition anymore due to casting errors.

[tsampazk]

On my earlier deleted response i had some things mixed up, sorry, this is actually unrelated to vision_opencv.

Maybe if you still have the original installation of the toolkit saved, you try this sudo apt-get install ros-foxy-vision-msgs first, before the rosdep install... command. I think that i just missed one or both of these commands in the instructions, or mistakenly thought that they where included somewhere in the links i provided, so i omitted them. It seems curious though, because i think that @ad-daniel and @passalis both ran this successfully without mentioning anything.

If even the above doesn't work maybe you missed the steps in the ROS2 installation regarding the keys, etc.:

1. sudo apt update && sudo apt install curl gnupg2 lsb-release
2. curl http://repo.ros2.org/repos.key | sudo apt-key add -
3. sudo sh -c 'echo "deb [arch=amd64,arm64] http://packages.ros.org/ros2/ubuntulsb_release -cs main" > /etc/apt/sources.list.d/ros2-latest.list'

from here steps 2 and 3 in the second section regarding ROS2 foxy installation.

If you get errors building the toolkit from scratch should be unrelated with this indeed.

[vniclas]

Running sudo apt-get install ros-foxy-vision-msgs solved it. Thanks! I also thought that this was included in the default installation.

[charsyme]

@tsampazk Is the licence of this file correct ?

opendr/projects/opendr_ws_2/src/opendr_perception/test/test_flake8.py

Line 1 in c28ae39

# Copyright 2017 Open Source Robotics Foundation, Inc.

My implementation in #291 is failing due to not containing the usual OpenDR licence.

[tsampazk]

@tsampazk Is the licence of this file correct ?

opendr/projects/opendr_ws_2/src/opendr_perception/test/test_flake8.py

Line 1 in c28ae39

# Copyright 2017 Open Source Robotics Foundation, Inc.

My implementation in #291 is failing due to not containing the usual OpenDR licence.

These are not OpenDR files so they contain their own licenses, which we shouldn't change. I think we could just delete them altogether, or add exceptions in the license testing as seen here for the simulation package also. Nevertheless, i think that we shouldn't put our own license on those files, as they are not are own.

[charsyme]

@tsampazk You are right, I didn't notice you added exceptions in checking the OpenDR licence to those files.

[ad-daniel]

As for the failure, from the error code it seems we are running once again out of RAM. I believe this tool was extended recently, which means two tests are occurring in the same job. We might need to split them further in the future

[tsampazk]

You beat me to it Daniel, thanks for the audio installation! 😆 I am now making the required changes in the readme.

[ad-daniel]

Works for me, thank you!

[ad-daniel]

This PR should be reviewed in priority so that I can finalize #353 before synchronizing on master. Some missing dependencies were shadowed by the fact the test release installed multiple tools even in the separate wheel test, rather than just installing engine+tool. It might have been necessary before but doesn't seem to be the case anymore. Additionally, some issues emerge just in this scenario (typically cases where no version limits are specified). Things work if another tool indirectly sets a limit, but would break when installed on it's own.

[passalis]

Overall seems fine to me! Thank you all for the hard work!