Record and Replay camera data with ROS2

• Orin AGX
• Orin NX 16

Test Metrics

• The Maximum SVO FPS metric indicates the mean recording frame rate achieved for each individual SVO file.
• The SVO file size metric indicates the mean file size of an SVO recorded over a recording period of 1 minute.

Replaying SVO files with the ZED Ros Wrapper #

Once you have recorded an SVO file, you can use the ZED ROS 2 wrapper to replay it, simulating a live camera stream. During SVO playback, you can enable or disable SDK modules—such as depth sensing, point cloud generation, and tracking—to extract additional data or test different processing pipelines. This flexibility allows you to experiment with various parameter settings and replay the same sequence multiple times, making it easier to debug, optimize, and validate your robotics applications without needing to re-record data.

To replay an SVO file, first set your desired parameters in the ZED wrapper’s configuration YAML file. Then, launch the wrapper with the following command:

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=<camera_model> svo_path:=</path/to/your/svo/file.svo2>

⚠️ At the moment, this feature is limited to single SVO replay.

To support debugging workflows involving SVO files within a robotics architecture, we provide several tools and tutorials:

• svo_control_node — A utility for precise control over SVO playback using the ZED ROS 2 wrapper. Features include pause/resume, step forward/backward, and playback speed adjustment.

• sync_node — Enables synchronized playback of SVO files alongside external navigation data from a ROS 2 bag. Supports pause/resume and step-wise playback, while maintaining time alignment.

• SVO to ROS Bag Conversion Workflow — A tutorial-based method to convert .svo files into standard ROS 2 bag files and merge them with navigation data from another bag, allowing unified playback and analysis.

👉 Follow the dedicated tutorial to learn how to use these tools effectively in your workflow.

Recording and replaying rosbag files #

Record rosbag files with the ZED ROS Wrapper #

Rosbag files provide a standard way to record and replay data within the ROS ecosystem. With the ZED ROS 2 Wrapper, you can capture not only camera sensor data but also any additional topics published by other nodes in your robotics stack. This makes rosbags ideal for debugging, analysis, and sharing complete system datasets. Recorded topics can be replayed and visualized using tools like RVIZ or Foxglove, enabling thorough inspection and troubleshooting of complex robotics applications.

✅ Main advantages of the Rosbag

• Record and replay the entire robotics stack, including all ROS topics published by any node in your system.
• Capture data from navigation modules, additional sensors, and both standard and custom (proprietary) ROS messages for complete dataset coverage.
• Easily share, analyze, and visualize recorded data using standard ROS tools and third-party applications.

🚫 Rosbag Limitations

• Recording large or high-frequency topics, such as images and point clouds, can quickly consume significant storage and system resources, potentially impacting performance.
• ZED parameters and processing options are set at recording time and cannot be changed during playback, limiting flexibility for post-recording optimization or experimentation.

Recording Best Practices #

To achieve optimal performance when working with ROS bags, we recommend following a set of best practices.

Use mcap storage #

The mcap format allows rosbags optimization and the capability to be immediately replayed through visualization tools like Foxglove.

$ sudo apt install ros-"$ROS_DISTRO"-rosbag2-storage-mcap

Use image_transport compression and ffmpeg codecs #

If you plan to record image topics, using FFmpeg with the NVENC (for desktop GPUs) or NvMpi (for Jetson boards) codecs in ROS 2 enables hardware-accelerated video encoding on NVIDIA GPUs. This offloads the computationally intensive encoding process from the CPU to the GPU, resulting in lower CPU usage and more stable system performance during recording. Leveraging GPU-based encoding helps maintain higher frame rates, reduces the risk of dropped frames, and allows for reliable rosbag recording even with multiple high-resolution video streams.

Follow the instructions here to build a version of ffmpeg that supports NVMPI:

git clone https://github.com/berndpfrommer/jetson-ffmpeg.git 
cd jetson-ffmpeg && mkdir build && cd build && cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr/local ..  && sudo make install 
sudo ldconfig && cd ../.. && git clone git://source.ffmpeg.org/ffmpeg.git -b release/7.1 --depth=1 
sudo apt install libx264-dev
cd jetson-ffmpeg && ./ffpatch.sh ../ffmpeg && cd ../ffmpeg && ./configure --enable-nonfree --enable-shared --enable-nvmpi --enable-gpl --enable-libx264  --prefix=/usr/local && sudo make install
echo 'export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH' >> ~/.bashrc && source ~/.bashrc

Once installed, ensure the codecs h264_nvmpi/hevc_nvmpi (for Jetson boards) or hevc_nvenc (for desktop GPU) are properly listed within the available ffmepg codecs :

ffmpeg -codecs

We recommend installing the image_transport plugins from source in your ROS workspace (remove any previously installed binary versions to avoid conflicts):

cd your_ros_ws/src/
# clone the image_transport_plugins package (use git commands to put it in the correct branch associated to your ros distro version)
git clone https://github.com/ros-perception/image_transport_plugins.git 
# clone ffmpeg packages to support the codes within ROS
git clone https://github.com/ros-misc-utilities/ffmpeg_image_transport.git
git clone https://github.com/ros-misc-utilities/ffmpeg_encoder_decoder.git
# [optional] clone the foxglove_compressed_video_transport to ensure proper Foxglove streaming
git clone https://github.com/ros-misc-utilities/foxglove_compressed_video_transport.git
#export ffmpeg library
cd ..
rosdep install --from-paths src --ignore-src -r -y
colcon build --cmake-args -DCMAKE_BUILD_TYPE=Release
source install/setup.bash

You should now be able to use the ffmpeg installed codecs with the ROS wrapper with the foxglove_compressed_video_transport package. To do so, modify slightly the zed camera launcher to add the chosen codec parameters for the desired type of images. For example, if you want to enable Foxglove image transport with the ffmpeg h264_nvmpi compression codec on the cameras rectified images topics, add these lines to the ZED wrapper node parameters :

node_parameters.append(
{
 #Add any desired parameters for the zed camera node, then add the images image compression codecs parameters
.' + camera_name_val + '.left.image_rect_color.foxglove.encoding': 'h264_nvmpi',
'.' + camera_name_val + '.left.image_rect_color.foxglove.profile': 'main',
'.' + camera_name_val + '.left.image_rect_color.foxglove.preset': 'medium',
'.' + camera_name_val + '.left.image_rect_color.foxglove.gop': 10,
'.' + camera_name_val +'.left.image_rect_color.foxglove.bitrate': 4194304,
'.' + camera_name_val + '.right.image_rect_color.foxglove.encoding': 'h264_nvmpi',
'.' + camera_name_val + '.right.image_rect_color.foxglove.profile': 'main',
'.' + camera_name_val + '.right.image_rect_color.foxglove.preset': 'medium',
'.' + camera_name_val + '.right.image_rect_color.foxglove.gop': 10,
'.' + camera_name_val +'.right.image_rect_color.foxglove.bitrate': 4194304
}
)

Use the following command to explore available codec parameters and options:

ffmpeg -h encoder=<encoder_codec_name>

You can then adjust the launcher configuration to match your desired recording setup. The parameters will be used at the next launch of the cameras, after recompiling your ros workspace containing the ZED wrapper.

Use rosbag recording as a component node available with IPC [Optional] #

Starting with ROS 2 Jazzy, the ros2 bag tool natively supports node composition. This allows you to launch the rosbag recorder as a composable node within a container, enabling efficient intra-process communication (IPC) and reducing data copying overhead.

For earlier ROS 2 distributions, you can achieve similar functionality using the rosbag2_composable_recorder community package. This package provides a component node for rosbag recording with IPC support, improving performance when recording high-bandwidth topics such as images or point clouds.

Tip: Using rosbag as a composable node with IPC is especially beneficial in multi-camera or high-throughput scenarios, as it minimizes CPU and memory usage during recording.

Recording Instructions #

1️⃣ Launch the ZED ROS 2 Wrapper: #

Single camera setup:

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=<camera_model>

Multi camera setup (up to 4 cameras):

ros2 launch zed_multi_camera zed_multi_camera.launch.py cam_names:='[zed1,...,zed4]' cam_models:='[zedx,...,zedxm]' cam_serials:='[<serial_camera_1>,...,<serial_camera_4>]'

2️⃣ Record node topics as a Rosbag mcap file #

ros2 bag record -s mcap /topic1 /topic2 ... 

In the example described above (recording each of the stereo camera images topics with foxglove_compressed_video_transport), the command would be:

ros2 bag record -s mcap /zed_multi/zed1/left/image_rect_color/foxglove /zed_multi/zed1/right/image_rect_color/foxglove /zed_multi/zed2/left/image_rect_color/foxglove /zed_multi/zed2/right/image_rect_color/foxglove ...

Recording Performances of the ZED Cameras images with the ZED Wrapper #

To evaluate recording performance, the metrics below are based on tests conducted with no additional SDK modules enabled (depth, positional tracking, object detection modules disabled), using up to four ZED X cameras connected to a single Jetson board. For each camera configuration, the results reflect the maximum achievable performance using the latest version of the ZED SDK. One Rosbag file is recorded with all the cameras left and right images topics. The test is launched with the h264_nvmpi codec added on each stereo camera rectified images topics with a bit_rate of 4194304 and medium preset.

Performance Tables #

Test Metrics

• The Maximum Rosbag FPS metric indicates the mean recording frame rate achieved for the rosbag file.
• The rosbag file size metric indicates the mean file size of a rosbag recorded over a period of 1 minute.

💡 Tips for recording rosbags efficiently and reduce overall recording load #

• Record only the topics you need.
• On Jetson, use the jetson_clocks.sh script.
• On Jetson, get the highest power mode possible (MAXN recommended)
• Split large bags with the --max-bag-size or --max-bag-duration parameters.
• Reduce the frame rate for the images and point clouds (e.g from 30 fps to 10 fps) to reduce rosbag loads.
• Reduce publishing rates of other topics when a fast publishing rate (>10 Hz) is not necessary.
• Images and point cloud sizes can be further reduced from the chosen resolution using the pub_downscale_factor parameter (e.g setting the parameter to 2.0 on a HD1200 resolution publishing images with a resolution of 960x608 pixels).
• The pub_frame_rate parameter clamps the publishing rate to the chosen limit.

Replaying the recorded rosbag #

You can replay your recorded mcap rosbag file using standard ROS 2 tools or visualize it directly in Foxglove. Foxglove natively supports the mcap format and compressed image topics via the foxglove-compressed-image-transport plugin, enabling efficient playback and inspection of high-bandwidth data. With Foxglove, you can pause, seek, and resume playback at any point, making it easy to analyze specific events or time ranges within your dataset.

To replay a rosbag in ROS 2, use:

ros2 bag play <your_bag_file>

For advanced playback options (such as rate control or topic remapping), refer to the ros2 bag play documentation.