Using Zenoh as ROS 2 Middleware

Zenoh is a pub/sub/query protocol that can be used as an alternative ROS 2 middleware through the rmw_zenoh implementation. Unlike the default DDS-based middleware, Zenoh communicates over TCP and relies on a lightweight router for discovery instead of UDP multicast.

This makes it a very good fit for the ZED ROS 2 nodes, which publish large messages such as high-resolution images and 3D point clouds:

No kernel buffer tuning required. Because Zenoh uses TCP, large messages are not split into UDP/IP fragments, so the sysctl network tuning that DDS needs to transfer point clouds and images is not required.
Discovery scales better. Nodes connect to a router instead of flooding the network with multicast discovery traffic, which helps in large systems and on networks where multicast is unreliable (e.g. Wi-Fi).
Shared memory can be enabled to move large messages between processes on the same machine without copying them over the network stack.

This guide applies to ROS 2 Humble and ROS 2 Jazzy. The commands are identical on both distributions and use the $ROS_DISTRO environment variable, which is set automatically to humble or jazzy when you source your ROS 2 setup file, so you can run them as they are.

ROS 2 messages include a type hash starting from Iron. rmw_zenoh embeds this hash in the Zenoh keys it uses to exchange data, so a Humble node and a Jazzy (or newer) node will discover each other but silently fail to communicate. Make sure every machine in your ROS 2 graph uses the same ROS 2 distribution when running over Zenoh.

Step 1 — Install rmw_zenoh

Pre-built binaries are available for Humble and Jazzy. Open a terminal console (Ctrl + Alt + t) and run:

$ sudo apt update
$ sudo apt install ros-$ROS_DISTRO-rmw-zenoh-cpp

The configuration described in this guide must be applied to every machine involved in the ROS 2 infrastructure that needs to send or receive ZED ROS 2 messages over Zenoh.

(Optional) Build from source

If you need the latest features or fixes, you can build rmw_zenoh from source instead:

$ mkdir -p ~/ws_rmw_zenoh/src && cd ~/ws_rmw_zenoh/src
$ git clone https://github.com/ros2/rmw_zenoh.git -b $ROS_DISTRO
$ cd ~/ws_rmw_zenoh
$ rosdep install --from-paths src --ignore-src --rosdistro $ROS_DISTRO -y
$ source /opt/ros/$ROS_DISTRO/setup.bash
$ colcon build --cmake-args -DCMAKE_BUILD_TYPE=Release

Remember to source the workspace (source ~/ws_rmw_zenoh/install/setup.bash) in every terminal where you want to use the source-built version.

Step 2 — Select Zenoh as the RMW implementation

ROS 2 chooses its middleware through the RMW_IMPLEMENTATION environment variable. Set it in every terminal where you launch a ROS 2 node (including the ZED node, the Zenoh router, and any subscriber such as RViz2):

$ export RMW_IMPLEMENTATION=rmw_zenoh_cpp

To apply it automatically to every new terminal, add the line to your ~/.bashrc file:

$ echo 'export RMW_IMPLEMENTATION=rmw_zenoh_cpp' >> ~/.bashrc

You can confirm that the correct middleware is selected with:

$ ros2 doctor --report | grep middleware

Step 3 — Start the Zenoh router

Unlike DDS, Zenoh disables UDP multicast discovery by default. Nodes discover each other through a Zenoh router (rmw_zenohd). You must start one router before launching any node (for multi-machine setups, see Running across multiple machines).

Open a dedicated terminal and run:

$ source /opt/ros/$ROS_DISTRO/setup.bash
$ export RMW_IMPLEMENTATION=rmw_zenoh_cpp
$ ros2 run rmw_zenoh_cpp rmw_zenohd

Leave this terminal open: the router must stay alive for the whole session. By default it listens on TCP port 7447.

By default, a node tries to reach the router only once at startup and continues even if it is not found. You can control this behavior with the ZENOH_ROUTER_CHECK_ATTEMPTS environment variable:

0 → wait indefinitely until a router is found.
a positive number N → try N times, once per second, then continue.
a negative number → skip the check entirely.

Setting export ZENOH_ROUTER_CHECK_ATTEMPTS=0 is handy to make sure your ZED node never starts publishing before the router is ready.

Step 4 — Launch the ZED node and verify

With the router running and RMW_IMPLEMENTATION exported, launch the ZED node as usual. For a stereo camera:

$ export RMW_IMPLEMENTATION=rmw_zenoh_cpp
$ ros2 launch zed_wrapper zed_camera.launch.py camera_model:=<camera_model>

Replace <camera_model> with your camera (e.g. zed2i, zedx, zedxm). See the ZED Stereo Node page for the full list of launch options.

In another terminal (with the same RMW_IMPLEMENTATION exported), check that the topics are available and that data flows:

$ export RMW_IMPLEMENTATION=rmw_zenoh_cpp
$ ros2 topic list
$ ros2 topic hz /zed/zed_node/point_cloud/cloud_registered

Quick sanity test

If you want to validate the Zenoh setup independently from the ZED node, use the standard ROS 2 demo nodes. With the router running, open two terminals:

$ # Terminal A
$ export RMW_IMPLEMENTATION=rmw_zenoh_cpp
$ ros2 run demo_nodes_cpp talker

$ # Terminal B
$ export RMW_IMPLEMENTATION=rmw_zenoh_cpp
$ ros2 run demo_nodes_cpp listener

The listener should print the messages published by the talker.

Running across multiple machines

A typical robotics setup has the ZED node running on the robot (e.g. an NVIDIA® Jetson™) and the visualization tools running on a remote workstation. With Zenoh, the recommended approach is to run one router per machine and connect the routers to each other.

On every machine, make sure all terminals use Zenoh (export RMW_IMPLEMENTATION=rmw_zenoh_cpp).
Start a router on each machine (Step 3).
Tell the workstation’s router to connect to the robot’s router. The quickest way is to start the router with a configuration override pointing at the robot’s IP address:

$ export RMW_IMPLEMENTATION=rmw_zenoh_cpp
$ export ZENOH_CONFIG_OVERRIDE='connect/endpoints=["tcp/<ROBOT_IP>:7447"]'
$ ros2 run rmw_zenoh_cpp rmw_zenohd

Replace <ROBOT_IP> with the IP address of the machine running the ZED node. Once the two routers are linked, every node on either machine can see all the topics.

Just like with DDS, all nodes that need to communicate must share the same ROS_DOMAIN_ID. rmw_zenoh honors this variable to isolate independent ROS 2 graphs running on the same network.

For a permanent setup, copy the default configuration file shipped with the package and edit its connect/listen endpoints instead of using the override variable:

$ cp $(ros2 pkg prefix rmw_zenoh_cpp)/share/rmw_zenoh_cpp/config/DEFAULT_RMW_ZENOH_ROUTER_CONFIG.json5 ~/zenoh_router_config.json5
$ export ZENOH_ROUTER_CONFIG_URI=~/zenoh_router_config.json5

When you point ZENOH_ROUTER_CONFIG_URI (router) or ZENOH_SESSION_CONFIG_URI (nodes) to a file, it replaces the default configuration, so always start from a copy of the default file to keep the recommended values.

Tuning Zenoh for large messages

ZED cameras publish large messages — high-resolution images and dense point clouds can each be several MB to tens of MB. Zenoh handles these far better than DDS out of the box, but a few settings help on demanding setups (4K images, high-density point clouds, constrained or wireless links).

No IP-fragmentation tuning needed

Because Zenoh transfers data over TCP, it is not affected by the UDP/IP fragmentation problems described in the DDS and Network Tuning page. You can skip the net.ipv4.ipfrag_* and net.core.rmem_max sysctl settings when using Zenoh. The default Zenoh configuration already accepts single messages up to 1 GiB (transport/link/rx/max_message_size), which is well above any ZED message size.

Prevent point clouds and images from being silently dropped

This is the most common issue when streaming ZED data over Zenoh, and it is worth understanding before anything else.

When network congestion occurs, Zenoh decides whether to drop a message or block the publisher until the message has been pushed out. Which behavior applies depends on the publisher’s ROS 2 QoS:

With KEEP_LAST history (the ZED nodes’ default: RELIABLE + KEEP_LAST, depth 10), Zenoh uses a drop strategy: if a message cannot be pushed to the network within wait_before_drop (1 ms by default), it is silently dropped. One millisecond is far too short for a multi-MB point cloud or a high-resolution image, so under load these messages can disappear without any error.
With RELIABLE + KEEP_ALL history, Zenoh uses a block strategy instead: the publisher waits until the message is fully sent, so nothing is dropped.

You have two ways to fix this (both confirmed by rmw_zenoh maintainers in issue #696):

Option A — Increase the drop timeout (recommended, no QoS change). Raise wait_before_drop (value in microseconds) in the terminal where you launch the ZED node, before starting it:

$ # Wait up to 1 second (1000000 µs) instead of 1 ms before dropping under congestion
$ export ZENOH_CONFIG_OVERRIDE='transport/link/tx/queue/congestion_control/drop/wait_before_drop=1000000'

This keeps the ZED nodes’ default QoS and lets large messages use the full bandwidth of the link.

Option B — Switch the publisher to RELIABLE + KEEP_ALL. Set the topic history to KEEP_ALL in the ZED node’s YAML configuration so Zenoh switches to the block strategy. The trade-off is that a slow subscriber or link can throttle the publisher’s rate. See the QoS profiles section for how to change these settings.

On low-bandwidth or wireless links, combining RELIABLE with TRANSIENT_LOCAL and a deep history can overwhelm the link when a late-joining subscriber requests the cached messages, eventually closing the transport (Unable to push non droppable network message ... Closing transport!). On such links, keep the history depth small and rely on the data-reduction techniques below.

Enable Shared Memory (SHM) for same-machine transfers

When the ZED node and its subscribers run on the same machine, shared memory lets Zenoh move large messages between processes without copying them through the network stack — a major win for point clouds and images.

SHM is disabled by default. To enable it, set the following override on the router and every node (publisher and subscribers):

$ export ZENOH_CONFIG_OVERRIDE='transport/shared_memory/enabled=true'

A few things to keep in mind:

The shared-memory segment defaults to 48 MiB per process. Make sure your host’s /dev/shm is large enough for all the processes involved.
Inside Docker, shared memory is isolated by default. Run your containers with --ipc=host (or a large enough --shm-size) so that SHM transport works across containers and with the host.
If a write to shared memory fails, Zenoh automatically falls back to the network transport, so enabling SHM is safe.
On startup you may see zenoh_shm::watchdog ... error setting scheduling priority warnings. These are benign (they relate to real-time thread scheduling) and can be safely ignored under normal conditions.

Increase the serialization buffer pool

rmw_zenoh recycles serialization buffers from a pool whose maximum size defaults to 8 MiB. Messages larger than this still work (they are allocated outside the pool), but raising the limit reduces allocations when streaming many large messages. Set it on the publisher side:

$ export RMW_ZENOH_BUFFER_POOL_MAX_SIZE_BYTES=33554432   # 32 MiB

Reduce the amount of data on the wire

The most effective tuning is often to send less data, especially over Wi-Fi. These techniques are middleware-agnostic and apply to Zenoh exactly as they do to DDS:

Use compressed image and point cloud topics — see Use compressed topics.
Publish smaller and less frequent data for previews — see Use smaller and less frequent information for data preview.

Useful environment variables

Variable	Purpose
`RMW_IMPLEMENTATION`	Selects the middleware. Set to `rmw_zenoh_cpp`.
`ZENOH_ROUTER_CHECK_ATTEMPTS`	How many times a node checks for the router at startup (see Step 3).
`ZENOH_CONFIG_OVERRIDE`	Override individual config keys, e.g. `key/path=value;key2/path=value2`. Used above to enable SHM and raise `wait_before_drop`.
`ZENOH_ROUTER_CONFIG_URI`	Path to a custom router configuration file.
`ZENOH_SESSION_CONFIG_URI`	Path to a custom node (session) configuration file.
`RMW_ZENOH_BUFFER_POOL_MAX_SIZE_BYTES`	Maximum serialization buffer pool size (default 8 MiB).
`RUST_LOG`	Zenoh log verbosity, e.g. `RUST_LOG=zenoh=info` for troubleshooting.

Congestion control and shared-memory transport for very large payloads are actively being improved upstream (finer-grained policies are planned for newer Zenoh releases). If you still hit dropped messages after applying the settings above, reduce the published resolution/frequency and follow the open reports on the rmw_zenoh issue tracker.

Switching back to DDS

To go back to the default DDS middleware, simply change (or unset) the RMW_IMPLEMENTATION variable and stop the Zenoh router:

$ # Back to Fast DDS (the ROS 2 default)
$ unset RMW_IMPLEMENTATION
$ 
$ # or, to use Cyclone DDS
$ export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp

Remember to also remove the corresponding line from ~/.bashrc if you added it. For DDS-specific tuning, refer to the DDS and Network Tuning page.