Skip to content

Commit ffb5336

Browse files
gabrikgabrik
committed
adding transcoding example
Signed-off-by: gabrik <gabriele.baldoni@gmail.com>
1 parent 56d126b commit ffb5336

File tree

11 files changed

+552
-0
lines changed

11 files changed

+552
-0
lines changed

‎README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,3 +44,11 @@ The purpose of this example is to demonstrate how Zenoh-Flow can handle a
4444
complex dataflow graph, like a robotic application.
4545

4646
Go to the [README](./montblanc/README.md) for instructions on how to run it.
47+
48+
49+
#### Transcoding
50+
51+
The purpose of this example is to demonstrate how Zenoh-Flow can handle be used within a Zenoh
52+
router to transcode live data.
53+
54+
Go to the [README](./transcoding/README.md) for instructions on how to run it.

‎transcoding/README.md‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
2+
# Zenoh-Flow for data transcoding.
3+
4+
This document will guide you in building, installing and configuring Zenoh-Flow together with Zenoh for data transcoding.
5+
6+
Note: this guide has been tested on Ubuntu 22.04 LTS
7+
## Prerequisites
8+
9+
In order to be able to build and run Zenoh-Flow the following dependencies are needed:
10+
11+
- build-essentials
12+
- python3-dev
13+
- python3-pip
14+
- python3-venv
15+
- clang
16+
- libclang-dev
17+
- rust
18+
- pkg-config
19+
20+
Please make sure those dependencies are installed before proceeding.
21+
22+
## Build Zenoh and Zenoh-Flow
23+
24+
Clone the repositories and build:
25+
```
26+
cd ~
27+
28+
git clone https://github.com/eclipse-zenoh/zenoh -b 0.7.2-rc
29+
cd zenoh
30+
cargo build --release --all-targets --features shared-memory
31+
32+
cd ..
33+
git clone https://github.com/eclipse-zenoh/zenoh-flow -b v0.5.0-alpha.1
34+
cd zenoh-flow
35+
cargo build --release --all-targets
36+
37+
cd ..
38+
git clone https://github.com/eclipse-zenoh/zenoh-flow-python -b v0.5.0-alpha.1
39+
cd zenoh-flow-python
40+
cargo build --release --all-targets
41+
42+
cd zenoh-flow-python
43+
44+
python3 -m venv venv
45+
source venv/bin/activate
46+
pip3 install -r requirements-dev.txt
47+
maturin build --release
48+
deactivate
49+
```
50+
51+
## Install
52+
53+
Install Zenoh and Zenoh-Flow
54+
55+
```
56+
cd ~
57+
58+
sudo mkdir -p /etc/zenoh/
59+
sudo mkdir -p /var/zenoh-flow/python
60+
sudo mkdir -p /var/zenoh-flow/flows
61+
sudo mkdir -p /etc/zenoh-flow/extensions.d
62+
63+
sudo cp zenoh/target/release/zenohd /usr/bin/
64+
sudo cp zenoh/target/release/libzenoh_plugin_*.so /usr/lib/
65+
66+
sudo cp zenoh-flow/target/release/libzenoh_plugin_zenoh_flow.so /usr/lib/
67+
sudo cp zenoh-flow/target/release/zfctl /usr/bin/
68+
sudo cp zenoh-flow-python/target/release/libzenoh_flow_python_*_wrapper.so /var/zenoh-flow/python
69+
sudo cp zenoh-flow-python/01-python.zfext /etc/zenoh-flow/extensions.d/
70+
sudo cp zenoh-flow/zfctl/.config/zfctl-zenoh.json /etc/zenoh-flow/
71+
72+
pip3 install ./zenoh-flow-python/target/wheels/eclipse_zenoh_flow-0.5.0a1-cp37-abi3-manylinux_2_34_x86_64.whl
73+
```
74+
75+
## Start Runtime
76+
77+
Copy the `zenoh-config.json` from this folder to `/etc/zenoh/zenoh.json`.
78+
79+
Now you can start the Zenoh router with the Zenoh-Flow plugin.
80+
Open a terminal and run: `RUST_LOG=debug zenohd -c /etc/zenoh/zenoh.json`
81+
82+
Then on another terminal run: `zfctl list runtimes`
83+
84+
You should get an output similar to this:
85+
```
86+
+----------------------------------+--------------------+--------+
87+
| UUID | Name | Status |
88+
+----------------------------------+--------------------+--------+
89+
| bb4a456d6c0948bfae21a6e8c9051d6b | protoc-client-test | Ready |
90+
+----------------------------------+--------------------+--------+
91+
```
92+
93+
This means that the zenoh-flow runtime is was loaded and it is ready.
94+
95+
## The transcoding application.
96+
97+
Copy the content of this folder in: `/var/zenoh-flow/flows` and run `pip3 install -r /var/zenoh-flow/flows/requirements.txt`.
98+
99+
100+
On a terminal start the publisher side: `cd /var/zenoh-flow/flows && python3 pub-proto.py`
101+
On a new terminal start the subscriber side: `cd /var/zenoh-flow/flows && python3 pub-cdr.py`
102+
103+
The subscriber will not receive any data as the transcoding is not yet deployed.
104+
105+
On a 3rd terminal instruct zenoh-flow to launch the transcoding flow: `zfctl launch /var/zenoh-flow/flows/dataflow.yml` it will return the instance id.
106+
107+
Now you should see the data being transcoded and received by your subscriber.
108+
109+
Once you are done you can list the current running flow instances: `zfctl list instances` and delete the running one with `zfctl destroy <instance uuid>`.
110+
111+
Once the instance is delete you will see that the subscriber is not going to receive any data.
112+
113+
114+
115+
116+
117+
118+
119+

‎transcoding/dataflow.yml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
flow: Transcoder
2+
vars:
3+
BASE_DIR: "/var/zenoh-flow/flows"
4+
5+
operators:
6+
- id : Conversion
7+
descriptor: "file://{{ BASE_DIR }}/transcoder.yml"
8+
sources:
9+
- id : ZenohSrc
10+
configuration:
11+
key-expressions:
12+
proto: data/proto
13+
descriptor: "builtin://zenoh"
14+
15+
sinks:
16+
- id : ZenohSink
17+
configuration:
18+
key-expressions:
19+
cdr: data/cdr
20+
descriptor: "builtin://zenoh"
21+
22+
links:
23+
- from:
24+
node : ZenohSrc
25+
output : proto
26+
to:
27+
node : Conversion
28+
input : in
29+
30+
- from:
31+
node : Conversion
32+
output : out
33+
to:
34+
node : ZenohSink
35+
input : cdr

‎transcoding/message.proto‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
syntax = "proto3";
2+
3+
message MyMsg {
4+
uint64 u_value = 1;
5+
string s_value = 2;
6+
}

‎transcoding/message_pb2.py‎

Lines changed: 34 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎transcoding/pub-proto.py‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
#
2+
# Copyright (c) 2022 ZettaScale Technology
3+
#
4+
# This program and the accompanying materials are made available under the
5+
# terms of the Eclipse Public License 2.0 which is available at
6+
# http://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
7+
# which is available at https://www.apache.org/licenses/LICENSE-2.0.
8+
#
9+
# SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
10+
#
11+
# Contributors:
12+
# ZettaScale Zenoh Team, <zenoh@zettascale.tech>
13+
#
14+
15+
import sys
16+
import time
17+
import argparse
18+
import itertools
19+
import json
20+
import zenoh
21+
from zenoh import config
22+
from message_pb2 import MyMsg
23+
24+
# --- Command line argument parsing --- --- --- --- --- ---
25+
parser = argparse.ArgumentParser(
26+
prog='z_pub',
27+
description='zenoh pub example')
28+
parser.add_argument('--mode', '-m', dest='mode',
29+
choices=['peer', 'client'],
30+
type=str,
31+
help='The zenoh session mode.')
32+
parser.add_argument('--connect', '-e', dest='connect',
33+
metavar='ENDPOINT',
34+
action='append',
35+
type=str,
36+
help='Endpoints to connect to.')
37+
parser.add_argument('--listen', '-l', dest='listen',
38+
metavar='ENDPOINT',
39+
action='append',
40+
type=str,
41+
help='Endpoints to listen on.')
42+
parser.add_argument('--key', '-k', dest='key',
43+
default='data/proto',
44+
type=str,
45+
help='The key expression to publish onto.')
46+
parser.add_argument('--value', '-v', dest='value',
47+
default='Pub from Python!',
48+
type=str,
49+
help='The value to publish.')
50+
parser.add_argument("--iter", dest="iter", type=int,
51+
help="How many puts to perform")
52+
parser.add_argument('--config', '-c', dest='config',
53+
metavar='FILE',
54+
type=str,
55+
help='A configuration file.')
56+
57+
args = parser.parse_args()
58+
conf = zenoh.Config.from_file(args.config) if args.config is not None else zenoh.Config()
59+
if args.mode is not None:
60+
conf.insert_json5(zenoh.config.MODE_KEY, json.dumps(args.mode))
61+
if args.connect is not None:
62+
conf.insert_json5(zenoh.config.CONNECT_KEY, json.dumps(args.connect))
63+
if args.listen is not None:
64+
conf.insert_json5(zenoh.config.LISTEN_KEY, json.dumps(args.listen))
65+
key = args.key
66+
value = args.value
67+
68+
# initiate logging
69+
zenoh.init_logger()
70+
71+
print("Opening session...")
72+
session = zenoh.open(conf)
73+
74+
print(f"Declaring Publisher on '{key}'...")
75+
pub = session.declare_publisher(key)
76+
77+
for idx in itertools.count() if args.iter is None else range(args.iter):
78+
time.sleep(1)
79+
80+
msg = MyMsg(
81+
u_value = idx,
82+
s_value = value
83+
)
84+
85+
print(f"Putting Data ('{key}': '{msg}')...")
86+
pub.put(msg.SerializeToString())
87+
88+
pub.undeclare()
89+
session.close()

‎transcoding/requirements.txt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
grpcio-tools==1.54.2
2+
pycdr2==1.0.0
3+
eclipse-zenoh==0.7.2rc0

0 commit comments

Comments
 (0)