Issues with using Zenoh_bridge_dds to route data between domains

[stevemcnamaraultra]

In the FAQ in states:

Question: How to communicate between DDS-domains?
Answer: Cyclone DDS relies on Zenoh for that

We have a need to route data from our system between two instances over the Internet. Our expectation is that the bridge would perform this function for us.

Upon trying to use the bridge though we have hit across what appear to be limitations of it. Note I say appears here because it is not completely clear if this is issues with our setup or the bridge itself.

1. The bridge does not route disposed or unregistered events. These are marked as not valid data in the data reader and the bridge ignores them. It only routes valid data. This causes issues with us trying to delete information within our system.
2. The bridge does not appear to implement DDS liveliness handling. A participant disappearing at one end does not cause an equivalent liveliness lost event at the other end of the link. This breaks consistency in our system as one system has information that should have been removed.

We are using the latest versions of Zenoh_bridge_dds and CycloneDDS both built by ourselves, all running on the same Ubuntu 22.04 instance. Our test setup is:

• DDS participants and Zenoh_bridge_dds in peer mode on domain 26
• DDS participants and Zenoh_bridge_dds in peer mode on domain 30
• Zenohd running

(Apologies for this information being a bit terse. I can add more detail if relevant)

I am hoping that we might be able to get some feedback around what the current limitations of the bridge are. Is what we are observing about its functionality correct or have we misunderstood something?

We would be happy to support extending the bridge if that would be useful though guidance about appropriate directions to go in could be useful so as to not disturb existing usages.

[eboasson]

I've asked around a bit and the issue seems to really belong to the zenoh bridge. I would appreciate if you could create the same issue there and then reference this one. Let's leave this one open for the time being, it might help other people.

It is holiday time in Europe so not everyone is around, but I think @gmartin82 and @JEnoch the best people to help you. I'm sure they'll pick the issue as soon as they are back.

[stevemcnamaraultra]

Thank you very much for the feedback.
I have opened a discussion issue on the Zenoh_bridge_dds page: eclipse-zenoh/roadmap#145.

[vivekpandey02]

Hi @stevemcnamaraultra, have you set forward_discovery=true in your zenoh_dds_bridge configuration?
When Forward_discovery is set to true, rather than creating a local route when discovering a local DDS entity, this discovery info is forwarded to the remote plugins/bridges. Those will create the routes, including a replica of the discovered entity.

I think it should work for you.

[stevemcnamaraultra]

Hi @vivekpandey02,
Thanks for the suggestion - any thoughts and suggestions are very much appreciated!
Unfortunately it has not improved the situation; the results appear very similar. I think I need to work up a simple test application that I could post here to demonstrate what I'm seeing.
For reference though here is what my test setup looks like:

Note I have tried to bridge in a combination of client and peer modes - after producing a simple test application my plan is to get a table of results for different combinations of the options.

[gmartin82]

Hi @stevemcnamaraultra. I believe the forward discovery mode is the most suitable mode for your use case but it's worth noting that the bridge doesn't actually forward the DDS discovery messages across the bridge. Instead compact Zenoh messages are sent between bridge instances indicating that readers or writers have been created or destroyed. This information is then used to create/destroy proxy DDS entities in remote bridge instances.

Regarding the two issues you highlighted:

1. The bridge doesn't understand instances of DDS topics so disposing or unregistering an instance won't be picked up in the remote domain unless the proxy entity writing that instance is destroyed. As you have observed the bridge only forwards valid data which is in raw CDR format.

2. A participant disappearing at one end should cause all proxy DDS entities created by the Zenoh DDS bridge related to that participant to be destroyed at the remote end. This should result in the remote DDS domain detecting the removal of these entities. Note however that participants aren't replicated in the remote domain and that the proxy DDS entities only support automatic Liveliness QoS.

[eboasson]

2. The bridge doesn't understand instances of DDS topics so disposing or unregistering an instance won't be picked up in the remote domain unless the proxy entity writing that instance is destroyed. As you have observed the bridge only forwards valid data which is in raw CDR format.

Surely that can be fixed by adding a few bits of metadata to each sample?

If I am overly optimistic and it would actually be a major project that doesn't fit well in the scope of the zenoh-DDS plugin, then an option would be to build one entirely based on Cyclone itself. It is not hard, just some tedious work to do the discovery and create the entities (the "ignore local" QoS setting should be sufficient to prevent feedback loops, the _guid variants of the entity creation functions would even allow retaining the GUIDs) and copy data between the two domains (with or without deserializing it in between). The TCP+TLS feature might come in handy as well, it isn't great, but it should do fine for this use case.

[gmartin82]

Speaking with @eboasson this should be possible by also forwarding the timestamp and statusinfo along with the data. Additionally 'dds_forwardcdr()' would need to be used instead of 'dds_writecdr()' to preserve these fields when republishing the data.

[stevemcnamaraultra]

Thank you @gmartin82 and @eboasson for the feedback - it heads towards the same conclusion that I have reached:

1. The zenoh_bridge_dds has limited functionality - it was designed for a specific use-case rather than as a fully featured bridge
2. The options either come down to extending it (whilst trying to not break the existing use-case) or writing a new bridge that directly uses CycloneDDS

Writing a new bridge that was completely CycloneDDS would probably be easiest as it would not entail me learning Rust. This would though negate the benefits of using the Zenoh routing algorithms over an unreliable network (the Internet).
I'll consider further before setting on a course of action but your feedback and time to discuss and respond is very much appreciated!

[eboasson]

@stevemcnamaraultra should you decide to dive into doing this completely with Cyclone, you'll probably run into some areas where the documentation is not-quite-excellent. Don't hesitate to ask!

Dynsub might be a good thing to look at first: the interpretation of the type objects that it does is not of any use to a bridge, but the discovery of endpoints and types and constructing topics on-the-fly based on the discovered types would be quite similar.

[stevemcnamaraultra]

We have decided to go in a different direction with this functionality and moved to supporting both CycloneDDS and Zenoh natively, selecting the middleware based on how we're deploying. As such we no longer need to progress producing a bridge between CycloneDDS and Zenoh.
Thank you for the feedback and comments - hopefully they will help anyone attempting this in the future.