Python API

[BasVolkers]

Hi, im looking to migrate my package to ROS2.
This package uses the python_bindings of rviz in ROS1.
Are these also available somewhere for ROS2? I can't seem to find them.

[clalancette]

We don't currently have Python bindings available for the ROS 2 version of RViz. I've assigned the issue to one of the rviz maintainers in ROS 2 to take a look at the feasibility of doing so. Likely this will need community contribution to do so, as it isn't on our current roadmap.

[jacobperron]

I don't know much about the Python bindings in ROS 1, but it looks like we are generating the Python API from the C++ code with shiboken and/or sip. So I think doing something similar should be feasible for ROS 2.

For anyone who is willing to tackle this, I'd rather see shiboken used for Python API generation over sip. This is mainly because of licensing (shiboken is LGPL, while sip is GPL). Furthermore, IIUC, Qt has officially adopted shiboken as well. It looks like the shiboken generator was disabled a long time ago in ROS 1, but I guess it doesn't mean much for ROS 2, since the C++ code is very different.

Another option for exposing a Python API could be to use pybind11. FWIW, we are using pybind11 for rosbag2 Python bindings and are currently in progress of refactoring rclpy to use pybind11.

This needs someone to take a deeper look, but I'd expect we might have an easier time using shiboken considering it actively being used by other Qt-based projects.

[EricCousineau-TRI]

See also: https://gitlab.kitware.com/autopybind11/autopybind11/-/blob/55f7b955a65622414ebd961325779e3c404c3e13/README.rst#alternatives-comparison

[hellantos]

I have made a first attempt with shiboken here: https://github.com/ipa-cmh/rviz/tree/add-python-bindings
It builds... Still some work and testing to though. Will open a PR when I am a little further along.

[friackazoid]

Hello, I want to port one python software to ROS2 and that is required python bindings for RVIZ. Can I help with this PR? what is required for the merge?

[hellantos]

@friackazoid there is #889 which works under circumstances. There is still a build dependency problem which I have only been able to solve for local build with python_qt_binding in same workspace. Also to merge Windows support is required. I have no idea where to start and test that.

I have a more recent version that we use in some internal projects - still throws a lot of warnings durign build, but works. Honestly, my funding for this activity got scrapped, so I cannot pursue it further right now. I will try and push the latest working version I have to #889.

Also check out ros-visualization/python_qt_binding#114 which documents my effort to update python_qt_bindings.

[KKSTB]

Thanks for your nice effort @ipa-cmh.

I would like to try VisualizationFramePy in my program. I downloaded the latest PR: https://github.com/ipa-cmh/rviz/tree/f324c63a5035c2351ddf599fe75e1995119ea89c

I tried to build it in this environment:
OS: Ububtu 22.04
ROS: iron
Build command: colcon build --merge-install --allow-overriding rviz2 rviz_default_plugins

But I am stuck at this colcon_cmake exception:

--- stderr: rviz_python_bindings
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/colcon_core/executor/init.py", line 91, in call
rc = await self.task(*args, **kwargs)
File "/usr/lib/python3/dist-packages/colcon_core/task/init.py", line 93, in call
return await task_method(*args, **kwargs)
File "/usr/lib/python3/dist-packages/colcon_ros/task/ament_cmake/build.py", line 59, in build
rc = await extension.build(
File "/usr/lib/python3/dist-packages/colcon_cmake/task/cmake/build.py", line 87, in build
rc = await self._reconfigure(args, env)
File "/usr/lib/python3/dist-packages/colcon_cmake/task/cmake/build.py", line 182, in _reconfigure
buildfile = get_buildfile(cmake_cache)
File "/usr/lib/python3/dist-packages/colcon_cmake/task/cmake/init.py", line 112, in get_buildfile
if 'Ninja' in generator:
TypeError: argument of type 'NoneType' is not iterable

At colcon_cmake/task/cmake/build.py", line 182, it indicates there is some sort of build environment reconfigure failure causing a non-zero return code (as indicated by the comments in colcon_cmake/task/cmake/build.py).

The colcon code:
https://github.com/colcon/colcon-cmake/blob/6c4985ba4cc0020ffbb4d2f357387917db1f121f/colcon_cmake/task/cmake/build.py#L182C14-L182C47
https://github.com/colcon/colcon-cmake/blob/6c4985ba4cc0020ffbb4d2f357387917db1f121f/colcon_cmake/task/cmake/__init__.py#L112C8-L112C42

I have installed python3_pyside2 and python3_qtpy (though not by rosdep, but by apt install python3-qtpy libpyside2-dev libshiboken2-dev python3-pyside2.qtcore python3-pyside2.qtgui python3-pyside2.qthelp python3-pyside2.qtnetwork python3-pyside2.qtprintsupport python3-pyside2.qtsvg python3-pyside2.qttest python3-pyside2.qtwidgets python3-pyside2.qtxml shiboken2 as indicated in https://github.com/ros/rosdistro/blob/master/rosdep/python.yaml)

Not sure what build environment is further required. Thanks.

[KKSTB]

I tried hard locating the problem using the verbose build command:
VERBOSE=1 colcon build --merge-install --allow-overriding rviz2 rviz_default_plugins --event-handlers console_cohesion+ --cmake-args -DCMAKE_VERBOSE_MAKEFILE=ON

and is able to locate the exception location (caused by ament_get_recursive_properties() at https://github.com/ipa-cmh/rviz/blob/f324c63a5035c2351ddf599fe75e1995119ea89c/rviz_python_bindings/CMakeLists.txt#L50).

I substituted ament_get_recursive_properties() with older version of include_dirs method in rviz_python_bindings CMakeLists.txt instead (https://github.com/ipa-cmh/rviz/blob/05f5a746b472aeadf2d58bb2c51b2198a8327b96/rviz_python_bindings/CMakeLists.txt#L83)

However, there is still a missing cmake function shiboken_generator_ext() at https://github.com/ipa-cmh/rviz/blob/f324c63a5035c2351ddf599fe75e1995119ea89c/rviz_python_bindings/CMakeLists.txt#L82.

Then I found this: https://github.com/ipa-cmh/python_qt_binding/tree/dd8e43fca94f4cd3f88952d4093270047da4c7cf.

After adding the last puzzle, the project can be built.

And I found I can further build with iron rviz2, by simply adding VisualizationFrame() to rviz_common (3e03e30)

[hellantos]

@KKSTB sorry mate, I was on vacation and it has been some time I worked on this, currently the funding has gone away again.
I think I have a version somewhere that will build directly, but it is not very nicely done and still produces a lot of errors.

In general, I had quite some problems with making this build stably, because the build cache and install folders usually contain some stuff that will make you miss some errors when doing changes.

In the end, we are not using rviz2 for end-user interfaces anymore. We now switched to nicegui.
https://github.com/zauberzeug/nicegui

[KKSTB]

Hi nice to hear from you @ipa-cmh.

I try to use the successfully built rviz_python_bindings in a freshly created ROS2 python package with an empty pyside window. It displays an empty window as intended, but it crashes with segmentation fault once I add a line to import rviz_shiboken:

Current thread 0x00007fc09e05e740 (most recent call first):
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1176 in create_module
  File "<frozen importlib._bootstrap>", line 571 in module_from_spec
  File "<frozen importlib._bootstrap>", line 674 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1078 in _handle_fromlist
  File "/home/kk/.local/lib/python3.10/site-packages/shiboken2/files.dir/shibokensupport/__feature__.py", line 142 in _import
...

Hope that there would be help on how to solve this issue.

[hellantos]

Try the following for import, that is how I have it in our project.

import qtpy
from qtpy.QtGui import *
from qtpy.QtCore import *

try:
    from qtpy.QtWidgets import *
except ImportError:
    pass
from rviz import YamlConfigReader, VisualizationFrame, Config, rviz_common

You can than do the following:

with open("parameters.yml", "w") as outfile:
  yaml.dump(param_file, outfile, default_flow_style=False)
frame = VisualizationFrame(str(Path("parameters.yml").absolute()))
frame.setApp(app)
frame.setSplashPath("")
frame.initialize()
sleep(3.0)
frame.loadDisplayConfig(file)

Finally you'll have to add the frame to your layout.

[KKSTB]

Thanks for your quick reply. I found that it is my system installation problem.

I found that if I use Qt classes in python_qt_binding that all rqt plugins are using, I still have to pip3 install PySide2 even though I have apt installed the pyside2 components.

If I freshly create ROS2 python package without following rqt plugin example, and use qtpy just as your example did, then the pip3 PySide2 is not required.

After pip uninstalling pyside2 there is no crash when importing. Will continue trying. Thanks @ipa-cmh!