Zenohex

Zenohex is Elixir API for Zenoh.

Zenoh is a new protocol for Zero Overhead Pub/Sub, Store/Query and Compute. The most obvious explanation is that Zenoh offers publication subscription-based communication capabilities. Within the same network, Zenoh can autonomously search for communication partner nodes like DDS. Between different networks, Zenoh can search for nodes through a broker (called a router in Zenoh) like MQTT. Also, Zenoh provides functions for database operations and computational processing based on the Key-Value Store. Moreover, it has plugins/bridges for interoperability with MQTT, DDS, REST, etc. as communication middleware, and influxdb, RocksDB, etc. as database stacks.

For more details about Zenoh, please refer to the official resources.

• Official Page
• GitHub
• Discord

Zenoh's core modules are implemented in Rust, but API libraries in various programming languages such as Python (zenoh-python), C (zenoh-c), C++ (zenoh-cpp) are officially provided.

So what we need is Elixir! With this library, you can call Zenoh from your Elixir application to perform its basic processing. This allows the creation and communication of a large number of fault-tolerant nodes with little memory load (we hope :D

Usage

Currently, Zenohex uses version 0.11.0 of Zenoh.

We recommend you use the same version to communicate with other Zenoh clients or routers since version compatibility is somewhat important for Zenoh. Please also check the description on Releases about the corresponding Zenoh version.

FYI, the development team currently uses the following versions.

• Elixir 1.17.3-otp-27
• Erlang/OTP 27.1.2
• Rust 1.82.0

Installation

zenohex is available in Hex.

You can install this package into your project by adding zenohex to your list of dependencies in mix.exs:

  defp deps do
    [
      ...
      {:zenohex, "~> 0.3.2"},
      ...
    ]
  end

Documentation is also available in HexDocs.

Zenohex can be also adapted to your Nerves application just by adding zenohex in mix.exs. Please refer to pojiro/zenohex_on_nerves as the example.

This repository uses Rustler to call Rust (Zenoh) modules from Elixir, and pre-compiled NIF modules are automatically downloaded at mix compile time (since v0.1.3). IOW, if you just want to use this library from your Elixir application, you do not need to prepare a Rust environment. If you still want to build Rust NIF modules locally, please refer to this section.

Getting Started

Zenohex has a policy of providing APIs that wrap the basic functionality of Zenoh like other API libraries.

Here is the first step to building an Elixir application and using this feature.

$ mix deps.get
$ mix compile
$ iex -S mix

iex()> {:ok, session} = Zenohex.open()
{:ok, #Reference<>}
iex()> {:ok, publisher} = Zenohex.Session.declare_publisher(session, "demo/example/test")
{:ok, #Reference<>}
iex()> {:ok, subscriber} = Zenohex.Session.declare_subscriber(session, "demo/**")
{:ok, #Reference<>}
iex()> Zenohex.Publisher.put(publisher, "Hello Zenoh Dragon")
:ok
iex()> Zenohex.Subscriber.recv_timeout(subscriber, 1000)
{:ok,
 %Zenohex.Sample{
   key_expr: "demo/example/test",
   value: "Hello Zenoh Dragon",
   kind: :put,
   reference: #Reference<>
 }}
iex()> Zenohex.Subscriber.recv_timeout(subscriber, 1000)
{:error, :timeout}

Practical examples

We implemented practical examples under the lib/zenohex/examples. Since they consist of Supervisor and GenServer, we think they are useful as examples of more Elixir-like applications.

Please read the lib/zenohex/examples/README.md to use them as your implementation's reference.

For developers

For most users, this section should be skipped.

Build NIF module locally

This subsection is for developers who want to build NIF module locally in your Elixir application or try to use this repository itself for the contribution (very welcome!!).

First, please install and configure the Rust environment according to the instructions on the official site.

Then, add the following to your config file (e.g., config/config.exs) or make sure it is added.

import Config

config :rustler_precompiled, :force_build, zenohex: true

When you want to build NIF module locally into your project, install Rustler by adding rustler to your list of dependencies in mix.exs:

  defp deps do
    [
      ...
      {:zenohex, "~> 0.3.2"},
      {:rustler, ">= 0.0.0", optional: true},
      ...
    ]
  end

Enhance mix test

This subsection describes some Tips for mix test.

The default mix test rebuilds the NIF module, so you can reduce the test time by doing the following in advance.

export API_OPEN_SESSION_DELAY=0 && mix compile --force

You can also reduce the test time by adjusting SCOUTING_DELAY as the follow.

SCOUTING_DELAY=30 mix test

This parameter is used to set the upper time limit for searching (scouting) for a communication peer node. The default value (when undefined) is 200 ms. IOW, if the test fails because the communication partner is not found within the set time, this value should be increased.

Finally, the default mix test only checks the communication of Zenoh nodes within the same session. If you wish to run communication tests between different sessions, please run the following.

USE_DIFFERENT_SESSION="1" mix test

FYI, CI does this in test-with-another-session. This test may fail on GHA depending on whether the scouting is successful or not.

Versions of dependencies

We think the correspondence between Zenoh (cargo crate) and Rustler is sensitive. Also, the version number of Rustler is specified in both mix.exs (Elixir hex package) and Cargo.toml (Rust cargo crate). Therefore, we clearly specify these version numbers with == in mix.exs and = in Cargo.toml.

How to release

These steps just follow the Recommended flow of rustler_precompiled.

1. Change versions, mix.exs, native/zenohex_nif/Cargo.toml
2. Run test, this step changes native/zenohex_nif/Cargo.lock version
3. Commit them and put the version tag, like v0.2.0
4. Puth the tag, like git push origin v0.2.0. This step triggers the .github/workflows/nif_precompile.yml
5. After the artifacts are made, run mix rustler_precompiled.download Zenohex.Nif --all to update checksum-Elixir.Zenohex.Nif.exs and commit it
6. Then publish to Hex

Resources

• Zenoh meets Elixir in Japan
  • presented in Zenoh User Meeting 2023 at 2023/12/12
  • SpeakerDeck
  • YouTube archive
• Zenohex - an eloquent, scalable and fast communication library for Elixir
  • presented in Code BEAM America 2024 at 2024/03/07
  • SpeakerDeck
  • YouTube archive

License

The source code of this repository itself is published under MIT License.
Please note that this repository mainly uses Zenoh which is licensed under Apache 2.0 and EPL 2.0 and Rustler which is licensed under either of Apache 2.0 or MIT.