Tutorial - Subscribe to ZED topics with NITROS in an Isaac™ ROS node
Tutorial - Subscribe to ZED topics with NITROS in an Isaac™ ROS node
This tutorial demonstrates how to subscribe to ZED topics using NITROS in an Isaac™ ROS node. It assumes you have already set up the ZED camera and are publishing its data using the zed-ros2-wrapper packages as described in the Setting up Isaac™ ROS to work with ZED Cameras in ROS 2 section.
The example explains how to create a ROS 2 component that subscribes to the ZED camera’s image and depth streams, using NITROS and performs benchmarking on the received data. The same component also subscribes to ZED camera’s image and depth streams using standard ROS 2 subscriptions and performs benchmarking on the same received data for comparison.
Setup the example
If you haven’t done so already, set up your development environment by following our NVIDIA® Isaac™ ROS installation guide.
Install the ZED ROS2 Examples packages in the workspace:
If you are using Docker, start the Isaac™ ROS environment with ZED support:
Build the Subscriber example application:
Create a ROS 2 Node with NITROS Subscriptions
The ROS 2 node is implemented in C++ as a ROS 2 component. The source code is located in the folder isaac_ros/zed_isaac_ros_nitros_sub/src/component.
The component is defined as a class ZedNitrosSubComponentthat inherits from rclcpp::Node.
The Nitros subscriber is declared as a member variable of the class in the file include/zed_nitros_sub_component.hpp:
the Nitros subscriber is then initialized in the function create_nitros_subscriber() of the class in the file src/zed_nitros_sub_component.cpp:
When creating the Nitros subscriber, we specify the topic name to subscribe to (image), and we must also specify the Nitros type of the topic.
The Nitros type must be one of the supported Nitros types of the ManagedNitrosSubscriber class. In this case, we are subscribing to either the image or depth topic of the ZED camera, which are of type sensor_msgs/msg/Image. The corresponding Nitros types are nitros_image_rgba8_t for the image topic and nitros_image_32FC1_t for the depth topic.
The Nitros type must be specified as a string, using the static member supported_type_name of the corresponding Nitros type class. It’s not possible to determine the Nitros type at runtime, so you must choose the correct type when creating the subscriber.
In the node we use a trick to determine at runtime whether we are subscribing to the image or depth topic. We use a parameter _isDepth that can be set by subscribing to the same topic using a standard ROS 2 subscription. When we receive the first message on the standard ROS 2 subscription, we check the encoding of the message to determine whether it’s an image or depth topic, and set the _isDepth parameter accordingly. This way, we can create the Nitros subscriber with the correct type:
The Nitros subscriber uses a callback function nitros_sub_callback() that is called whenever a new message is received on the subscribed topic. The callback function receives a NitrosImageView object that contains the image data and metadata:
How the example works
The example node subscribes to the ZED camera’s image or depth topics using both NITROS and standard ROS 2 subscriptions. It performs benchmarking on the received data to compare the performance of NITROS and standard ROS 2 subscriptions.
At the start of the node, we create a standard ROS 2 subscription to the image topic and receive a number of messages as set by the parameter benchmark.tot_samples. We then perform benchmarking on the received messages to measure:
- The average latency of the messages with minimum and maximum latency and standard deviation.
- The average frame rate of the messages with minimum and maximum frame rate and standard deviation.
- The average CPU usage of the node performing the subscription process with minimum and maximum CPU usage and standard deviation.
- The average GPU usage of the node performing the subscription process with minimum and maximum GPU usage and standard deviation.
When the benchmarking is complete, we unsubscribe from the standard ROS 2 subscription and create the NITROS subscriber to the same topic and repeat the benchmarking process.
Finally, when we received benchmark.tot_samples messages with the NITROS subscriber, we print the benchmarking results, and compare the performance of NITROS and standard ROS 2 subscriptions.
The results of the benchmarking are printed to the console and saved to a CSV file defined by the parameter benchmark.csv_log_file.
Advanced configuration of the example
The folder config contains a YAML file named example_params.yaml that defines the parameters of the example.
The section general is used to overwrite the setup of the ZED ROS2 Wrapper node:
grab_resolution: The native camera grab resolution. ‘HD1200’, ‘HD2K’, ‘HD1080’, ‘HD720’, ‘SVGA’, ‘VGA’, ‘AUTO’.grab_frame_rate: ZED SDK internal grabbing rate (FPS).pub_resolution: The resolution used for image and depth map publishing. ‘NATIVE’ to use the samegeneral.grab_resolution-CUSTOMto apply thegeneral.pub_downscale_factordownscale factory to reduce bandwidth in transmission.pub_downscale_factor: Rescale factor used to rescale image before publishing when ‘pub_resolution’ is ‘CUSTOM’rescale factor used to rescale image before publishing when ‘pub_resolution’ is ‘CUSTOM’.pub_frame_rate: Frequency of publishing of visual images and depth data. This value must be equal or less than the camera framerate (grab_frame_rate).
The section benchmark is used to configure the benchmarking process:
tot_samples: Total number of samples to process.cpu_gpu_load_period: Period in milliseconds to retrieve CPU and GPU load statistics.cpu_gpu_load_avg_wnd_size: Number of samples to use to compute the average CPU and GPU load average.csv_log_file: If not empty, log the benchmark results in a CSV file with the specified name.
The section debug is used to debug the nodes:
debug_nitros: Enable Nitros debug information.use_pub_timestamps: Used to measure the real communication latency. If true, the latency is measured by setting the timestamps in the header of the messages to the system time just before publishing. If false, the latency is measured using the ZED camera timestamps, when the data is grabbed from the camera. This method allows to retrieve the end-to-end latency, the full time it takes for a message to travel from the camera buffer to the subscriber.
Run the example
First, make sure you compiled and built the example as described in the Setup section.
If you are using Docker, make sure to start the Isaac™ ROS environment with ZED support:
To launch the example, you must first select the topic to subscribe to. The launch file requires you to specify the topic name using the argument topic_name. For example, to subscribe to the image topic, use topic_name:=/zed_isaac/zed/rgb/rect/image, or to subscribe to the depth topic, use topic_name:=/zed_isaac/zed/depth/depth_registered.
for example, to subscribe to the RGB image topic of a ZED X One GS camera:
If you list all the topics published by the ZED ROS2 Wrapper node using the command ros2 topic list, you will see that all the image and depth topics have also a suffix /nitros. You do not need to specify that suffix when subscribing to the topics using NITROS, as the Managed NITROS subscriber automatically subscribes to the Nitros topic when possible. You can confirm this by looking at the ROS 2 Node Graph using the command ros2 run rqt_graph rqt_graph.
Example output
The example will print the benchmarking results to the console when the benchmarking is complete.
We executed the example on an NVIDIA® Jetson AGX Orin, using a ZED X One GS camera running at HD1200 @ 60 FPS.
The console output will look something like this:
If you specified a CSV log file in the parameters, the benchmarking results will be saved to the specified file. If you set it as a filename, without the path, the file will be saved in the ISAAC_ROS_WS directory.
This is an example of the contents of the CSV log file:
The output shows the benchmarking results for both the DDS subscriber and the Nitros subscriber, including the average, minimum, maximum, and standard deviation for each metric. It also includes a comparison of the two subscribers for each metric.
For example, in the output above, we can see that the average latency of the DDS subscriber is 0.00477946 sec, while the average latency of the Nitros subscriber is 0.000404906 sec. This means that the Nitros subscriber has a significantly lower latency than the DDS subscriber, 1080.39% lower in this case.
While the frame rate of the DDS subscriber is slightly higher than that of the Nitros subscriber, the difference is minimal (0.64333% higher for DDS), the standard deviation is also higher for the DDS subscriber, indicating that the frame rate is more stable with the Nitros subscriber.
The launch file explained
The launch file is located in the folder launch and is named zed_isaac_ros_nitros_sub.launch.py.
The launch file is responsible for starting all the necessary nodes and configurations for example. It sets up the parameters, including the ZED camera settings and the benchmarking parameters, and launches the required nodes in a ROS 2 component container.
Retrieve the path to the configuration file:
Set the namespace and define the launch arguments:
Create the ROS 2 component container to run the nodes:
Call the ZED ROS2 Wrapper launch file to start the ZED camera node, passing the necessary arguments, including the camera model, namespace, and parameters file:
The enable_ipc argument is set to false because the Nitros subscriber does not support IPC transport.** This is a mandatory setting when using NITROS**.
Add the ZED Isaac NITROS Subscriber component to the container, passing the necessary arguments, and remapping the topic name to subscribe to:
Again, the use_intra_process_comms argument is set to False because the Nitros subscriber does not support intra-process communication. This is a mandatory setting when using NITROS.
Finally, load the subscriber component into the container:
Conclusion
This tutorial demonstrated how to subscribe to ZED camera topics using NITROS in an Isaac™ ROS node. We created a ROS 2 component that subscribes to the ZED camera’s image and depth streams using both NITROS and standard ROS 2 subscriptions, and performed benchmarking on the received data to compare the performance of the two methods.
The results showed that the NITROS subscriber has significantly lower latency compared to the standard ROS 2 subscription, while maintaining a comparable frame rate. Additionally, the CPU and GPU usage were also lower for the NITROS subscriber, indicating better efficiency.
The values of the standard deviation for the Nitros subscriber are also lower, indicating a more stable performance.
This example can be used as a starting point for integrating ZED cameras into your Isaac™ ROS applications, leveraging the performance benefits of NITROS for high-throughput data streams.