ROS 2 - ZED Node

The easiest way to start a ZED ROS2 node is using the command line:

ZED:

$ ros2 launch zed_wrapper zed.launch.py

ZED Mini:

$ ros2 launch zed_wrapper zedm.launch.py

ZED 2:

$ ros2 launch zed_wrapper zed2.launch.py

Published Topics

The ZED node publishes data to the following topics:

  • Left camera

    • ~/left/camera_info: Left camera calibration data.
    • ~/left/image_rect_color: Left camera color rectified image.
    • ~/left/image_rect_gray: Left camera gray rectified image.
    • ~/left_raw/camera_info: Left camera raw calibration data.
    • ~/left_raw/image_rect_color: Left camera color unrectified image.
    • ~/left_raw/image_rect_gray: Left camera gray unrectified image.
    • ~/rgb/camera_info: RGB calibration data.
    • ~/rgb/image_rect_color: RGB color rectified image.
    • ~/rgb/image_rect_gray: RGB gray rectified image.
    • ~/rgb_raw/camera_info: RGB raw calibration data.
    • ~/rgb_raw/image_rect_color: RGB color unrectified image.
    • ~/rgb_raw/image_rect_gray: RGB gray unrectified image.
  • Right camera

    • ~/right/camera_info: Right camera calibration data.
    • ~/right/image_rect_color: Right camera color rectified image.
    • ~/right/image_rect_gray: Right camera gray rectified image.
    • ~/right_raw/camera_info: Right camera raw calibration data.
    • ~/right_raw/image_rect_color: Right camera color unrectified image.
    • ~/right_raw/image_rect_gray: Right camera gray unrectified image.
  • Stereo pair

    • ~/stereo/image_rect_color: side-by-side left/right rectified stereo pair. Calibration data are available in the topics: ~/left/camera_info and ~/right/camera_info
    • ~/stereo_raw/image_raw_color: side-by-side left/right unrectified stereo pair. Calibration data are available in the topics: ~/left_raw/camera_info and ~/right_raw/camera_info
  • Depth and point cloud

    • ~/depth/camera_info: Depth camera calibration data
    • ~/depth/depth_registered: Depth map image registered on left image
    • ~/point_cloud/cloud_registered: Registered color point cloud
    • ~/confidence/confidence_map: Confidence image (doubleing point values)
    • ~/disparity/disparity_image: Disparity image
  • Sensors data

    • ~/left_cam_imu_transform: Static transform from left camera to IMU sensor
    • ~/imu/data: Accelerometer, gyroscope, and orientation data in Earth frame
    • ~/imu/data_raw: Accelerometer and gyroscope data in Earth frame
    • ~/imu/mag: Raw magnetometer data (only ZED2)
    • ~/atm_press: atmospheric pressure (only ZED2)
    • ~/temperature/imu: temperature measured by the IMU sensor
    • ~/temperature/left: temperature measured near the left CMOS sensor
    • ~/temperature/right: temperature measured near the right CMOS sensor
  • Positional tracking

    • ~/pose: Absolute 3D position and orientation relative to the Map frame (Sensor Fusion algorithm + SLAM)
    • ~/pose_with_covariance: Camera pose referred to Map frame with covariance
    • ~/odom: Absolute 3D position and orientation relative to the Odometry frame (pure visual odometry for ZED, visual-inertial odometry for ZED-M)
    • ~/path_map: Sequence of camera poses in Map frame
    • ~/path_odom: Sequence of camera odometry poses in Map frame
  • Mapping

    • ~/mapping/fused_cloud: fused point cloud created when the enable_mapping service is called
  • Object Detection

    • ~/obj_det/objects: object detected frame by frame when the enable_obj_det service is called

Note: Each topic has a common prefix created by the launch file as ~ = /<namespace>/<node_name>/, where <namespace> is replaced with the value of the parameter camera_name and <node_name> is the name of the node, e.g. zed_node.

Image Transport

The ROS2 wrapper supports the stack image_transport introduced with ROS2 Crystal Clemmys.

All the image topics are published using the image_transport::CameraPublisher object, that correctly associates the sensor_msgs::msg::CameraInfo message to the relative sensor_msg::msg::Image message and creates the relative compressed image streams.

QoS profiles

All the topics are configured to use the following ROS2 QoS profile:

  • History: KEEP_LAST
  • Depth: 1
  • Reliability: RELIABLE
  • Durability: VOLATILE

The QoS settings can be modified changing the relative parameters in the YAML files, as described below.

When creating a subscriber, be sure to use a compatible QOS profile according to the following tables:

Compatibility of QoS durability profiles:

Publisher Subscriber Connection Result
Volatile Volatile Yes Volatile
Volatile Transient local No -
Transient local Volatile Yes Volatile
Transient local Transient local Yes Transient local

Compatibility of QoS reliability profiles:

Publisher Subscriber Connection Result
Best effort Best effort Yes Best effort
Best effort Reliable No -
Reliable Best effort Yes Best effort
Reliable Reliable Yes Reliable

In order for a connection to be made, all of the policies that affect compatibility must be compatible. For instance, if a publisher-subscriber pair has compatible reliability QoS profiles, but incompatible durability QoS profiles, the connection will not be made.

Configuration parameters

Specify your launch parameters in the common.yaml, zed.yaml, zedM.yaml and zed2.yaml files available in the folder zed_wrapper/config.

General parameters

Namespace: general

common.yaml

Parameter Description Value
svo_file Specify SVO filename string, Note: relative file paths are not allowed
svo_loop Restart the SVO if the end of the file is reached true, false
svo_realtime if true SVO will be played trying to respect the original framerate eventually skipping frames, otherwise every frame will be processed respecting the pub_frame_rate setting true, false
debug_mode Enable debug log outputs true, false
camera_timeout_sec Camera timeout (sec) if communication fails int
camera_max_reconnect Reconnection temptatives after timeout, before shutting down int
camera_flip Flip the camera data if it is mounted upsidedown true, false
zed_id Select a ZED camera by its ID. IDs are assigned by Ubuntu. Useful when multiple cameras are connected. ID is ignored if an SVO path is specified int, default 0
serial_number Select a ZED camera by its Serial Number int
resolution Set the ZED camera resolution 0: HD2K, 1: HD1080, 2: HD720, 3: VGA
sdk_verbose Enable/disable the verbosity of the SDK true, false
grab_frame_rate Set ZED camera grab framerate int
pub_frame_rate* Set the video/depth publish rate double
gpu_id Select a GPU device for depth computation int

* Dynamic parameter

zed.yaml, zedm.yaml and zed2.yaml

Parameter Description Value
camera_model Type of Stereolabs camera string, zed, zedm or zed2
camera_name User name for the camera, can be different from camera model string

Video parameters

Namespace: video

common.yaml

Parameter Description Value
extrinsic_in_camera_frame if false extrinsic parameter in camera_info will use ROS native frame (X FORWARD, Z UP) instead of the camera frame (Z FORWARD, Y DOWN) [true use old behavior as for version < v3.1] true, false
img_downsample_factor Resample factor for image data matrices. The SDK works with native data sizes, but publishes rescaled matrices double, [0.01,1.0]
brightness* Image brightness int, [0,8]
contrast* Image contrast int, [0,8]
hue* Image hue int, [0,11]
saturation* Image saturation int, [0,8]
sharpness* Image sharpness int, [0,8]
gamma* Image gamma int, [0,8]
auto_exposure_gain* Enable the auto exposure and auto gain true, false
exposure* Exposure value if auto-exposure is false int [0,100]
gain* Gain value if auto-exposure is false int [0,100]
auto_whitebalance* Enable the auto white balance true, false
whitebalance_temperature* White balance temperature int [28,65]
qos_history How to queue messages 0: KEEP_LAST - 1: KEEP_ALL
qos_depth Size of the queue if KEEP_LAST int
qos_reliability Enable resending of not delivered message, if RELIABLE 0: BEST_EFFORT - 1: RELIABLE
qos_durability Enable storing of unread message if TRANSIENT_LOCAL 0: TRANSIENT_LOCAL - 1: VOLATILE

* Dynamic parameter

Depth parameters

Namespace: depth

common.yaml

Parameter Description Value
quality Select depth map quality 0: NONE, 1: PERFORMANCE, 2: MEDIUM, 3: QUALITY, 4: ULTRA
sensing_mode Select depth sensing mode 0: STANDARD, 1: FILL
depth_stabilization Enable depth stabilization. Stabilizing the depth requires an additional computation load as it enables tracking true, false
openni_depth_mode Enable/disable the depth OpenNI format (16 bit, millimeters) int, [0,1]
depth_downsample_factor Resample factor for depth data matrices. The SDK works with native data sizes, but publishes rescaled matrices double, [0.01,1.0]
point_cloud_freq* frequency of the pointcloud publishing (equal or less to grab_frame_rate value) double
depth_confidence* Depth confidence threshold (lower values = more filtering) int [0,100]
depth_texture_conf* Depth tectureconfidence threshold (lower values = more filtering) int [0,100]
qos_history How to queue messages 0: KEEP_LAST - 1: KEEP_ALL
qos_depth Size of the queue if KEEP_LAST int
qos_reliability Enable resending of not delivered message, if RELIABLE 0: BEST_EFFORT - 1: RELIABLE
qos_durability Enable storing of unread message if TRANSIENT_LOCAL 0: TRANSIENT_LOCAL - 1: VOLATILE

Note: To disable depth calculation set quality to NONE. All the other parameters relative to depth will be ignored and only the video streams and sensors data (if using a ZED-M and ZED2) will be available

* Dynamic parameter

zed.yaml, zedm.yaml and zed2.yaml

Parameter Description Value
min_depth Minimum depth value (from the camera) that will be computed double
max_depth Maximum range allowed for depth. Values beyond this limit will be reported as TOO_FAR double, ]0.0,20.0]

* Dynamic parameter

Positional tracking parameters

Namespace: tracking

common.yaml

Parameter Description Value
publish_tf Enable publishing of transform odom -> base_link and all the other transforms true, false
publish_map_tf Enable publishing of transform map -> odom. Only valid if publish_tf is true true, false
base_frame Frame_id of the base frame string
map_frame Frame_id of the pose message string
odometry_frame Frame_id of the odom message string
area_memory_db_path The full path of the space memory DB string
pose_smoothing Enable smooth pose correction for small drift correction true, false - Enable only if making an AR application
area_memory Enable Loop Closing algorithm true, false
floor_alignment Indicates if the floor must be used as origin for height measures true, false
two_d_mode Enable if robot moves on a planar surface. Forces Z to fixed_z_value, roll and pitch to zero true, false
fixed_z_value Fixed value assigned to Z coordinate if two_d_mode is true double
initial_base_pose Initial reference pose. Useful if the base_link does not start from the origin array
path_pub_rate Frequency (Hz) of publishing of the path messages double
path_max_count Maximum number of poses kept in the pose arrays (-1 for infinite) int
qos_history How to queue messages 0: KEEP_LAST - 1: KEEP_ALL
qos_depth Size of the queue if KEEP_LAST int
qos_reliability Enable resending of not delivered message, if RELIABLE 0: BEST_EFFORT - 1: RELIABLE
qos_durability Enable storing of unread message if TRANSIENT_LOCAL 0: TRANSIENT_LOCAL - 1: VOLATILE

Sensors parameters (ZED-M and ZED2 only)

Namespace: imu

zedm.yaml and zed2.yaml

Parameter Description Value
sensors_image_sync Synchronize Sensors messages with latest published video/depth message true, false
qos_history How to queue messages 0: KEEP_LAST - 1: KEEP_ALL
qos_depth Size of the queue if KEEP_LAST int
qos_reliability Enable resending of not delivered message, if RELIABLE 0: BEST_EFFORT - 1: RELIABLE
qos_durability Enable storing of unread message if TRANSIENT_LOCAL 0: TRANSIENT_LOCAL - 1: VOLATILE

Object detection parameters (ZED2 only)

Namespace: object_detection

zed2.yaml

Parameter Description Value
od_enabled Automatically enables Object Detection when the node starts true, false
confidence_threshold* Minimum confidence for an object to be added to the detected list double [0,100]
model Detection model 0: MULTI_CLASS_BOX - 1: MULTI_CLASS_BOX_ACCURATE - 2: HUMAN_BODY_FAST - 3: HUMAN_BODY_ACCURATE
mc_people* Enable people detection for MULTI_CLASS_BOX and MULTI_CLASS_BOX_ACCURATE models true, false
mc_vehicle* Enable vehicles detection for MULTI_CLASS_BOX and MULTI_CLASS_BOX_ACCURATE models true, false
mc_bag* Enable bags detection for MULTI_CLASS_BOX and MULTI_CLASS_BOX_ACCURATE models true, false
mc_animal* Enable animals detection for MULTI_CLASS_BOX and MULTI_CLASS_BOX_ACCURATE models true, false
mc_electronics* Enable electronic devices detection for MULTI_CLASS_BOX and MULTI_CLASS_BOX_ACCURATE models true, false
mc_fruit_vegetable* Enable fruits and vegetables detection for MULTI_CLASS_BOX and MULTI_CLASS_BOX_ACCURATE models true, false
body_fitting Enable/disable body fitting for HUMAN_BODY_FAST and HUMAN_BODY_ACCURATE models true,false
qos_history How to queue messages 0: KEEP_LAST - 1: KEEP_ALL
qos_depth Size of the queue if KEEP_LAST int
qos_reliability Enable resending of not delivered message, if RELIABLE 0: BEST_EFFORT - 1: RELIABLE
qos_durability Enable storing of unread message if TRANSIENT_LOCAL 0: TRANSIENT_LOCAL - 1: VOLATILE

* Dynamic parameter

Mapping parameters

Namespace: mapping

Parameter Description Value
mapping_enabled Enable/disable the mapping module true, false
resolution Resolution of the fused point cloud [0.01, 0.2] double, default=0.1
max_mapping_range Maximum depth range while mapping in meters (-1 for automatic calculation) [2.0, 20.0] double
fused_pointcloud_freq Publishing frequency (Hz) of the 3D map as fused point cloud double
qos_history How to queue messages 0: KEEP_LAST - 1: KEEP_ALL
qos_depth Size of the queue if KEEP_LAST int
qos_reliability Enable resending of not delivered message, if RELIABLE 0: BEST_EFFORT - 1: RELIABLE
qos_durability Enable storing of unread message if TRANSIENT_LOCAL 0: TRANSIENT_LOCAL - 1: VOLATILE

Dynamic parameters

The ZED node lets you reconfigure many parameters dynamically during the execution of the node. All the dynamic parameters are indicated with a * in the list above and with the tag [DYNAMIC] in the comments of the YAML files.

You can set the parameters using the CLI command ros2 param set, e.g.:

$ ros2 param set /zed/zed_node depth.confidence 80

if the parameter is set successfully, you will get a confirmation message:

Set parameter successful

In the case you tried to set a parameter that’s not dynamically reconfigurable, or you specified an invalid value, you will get this error:

$ ros2 param set /zed/zed_node depth.confidence 150
Set parameter failed

and the ZED node will report a warning message explaining the error type:

1538556595.265117561: [zed.zed_node] [WARN]	The param 'depth.confidence' requires an INTEGER value in the range ]0,100]

Transform frame

The ZED ROS2 wrapper broadcasts multiple coordinate frames that each provides information about the camera’s position and orientation. If needed, the reference frames can be changed in the launch file.

  • base_frame is the current position and orientation of the reference base of the robot
  • <camera_name>_camera_center is the current position and orientation of ZED, determined by visual odometry and the tracking module
  • <camera_name>_right_camera is the position and orientation of the ZED’s right camera
  • <camera_name>_right_camera_optical is the position and orientation of the ZED’s right camera optical frame
  • <camera_name>_left_camera is the position and orientation of the ZED’s left camera
  • <camera_name>_left_camera_optical is the position and orientation of the ZED’s left camera optical frame
  • <camera_name>_imu_link is the origin of the inertial data frame (ZED-M and ZED2 only)
  • <camera_name>_mag_link is the origin of the magnetometer frame (ZED2 only)
  • <camera_name>_baro_link is the origin of the barometer frame (ZED2 only)
  • <camera_name>_temp_left_link is the origin of the left temperature frame (ZED2 only)
  • <camera_name>_temp_right_link is the origin of the right temperature frame (ZED2 only)

For RVIZ compatibilty, the root frame map_frame is called map. The TF tree generated by the zed_wrapper reflects the standard descripted in REP105. The odometry frame is updated using only the “visual odometry” information. The map frame is updated using the Tracking algorithm provided by the Stereolabs SDK, fusing the inertial information from the IMU sensor if using a ZED-M camera.

map_frame (`map`)
└─odometry_frame (`odom`)
   └─base_frame (`base_link`)
      └─camera_frame (`<camera_name>_camera_center`)
      |  └─left_camera_frame (`<camera_name>_left_camera_frame`)
      |  |     └─left_camera_optical_frame (`<camera_name>_left_camera_optical_frame`)
      |  |     └─left_temperature:frame (`<camera_name>_temp_left_link`)
      |  └─right_camera_frame (`<camera_name>_right_camera_frame`) (*only ZED 2*)
      |        └─right_camera_optical_frame (`<camera_name>_right_camera_optical_frame`)
      |        └─left_temperature:frame (`<camera_name>_temp_left_link`)
      └─imu_frame (`<camera_name>_imu_link`) (*only ZED-M and ZED 2*)
      └─magnetometer_frame (`<camera_name>_mag_link`) (*only ZED 2*)
      └─barometer_frame (`<camera_name>_baro_link`) (*only ZED 2*)

Services

The ZED node provides the following services:

  • ~/set_pose: Restarts the Tracking algorithm setting the initial pose of the camera to the value passed as vector parameter -> [X, Y, Z, R, P, Y]
  • ~/reset_pos_tracking: Restarts the Tracking algorithm setting the initial pose to the value available in the param server or to the latest pose set with the service set_pose
  • ~/reset_odometry: Resets the odometry values eliminating the drift due to the Visual Odometry algorithm. The new odometry value is set to the latest camera pose received from the tracking algorithm
  • ~/start_svo_rec: Start recording an SVO file. If no filename is provided the default zed.svo is used. If no path is provided with the filename the default recording folder is ~/.ros/
  • ~/stop_svo_rec: Stop an active SVO recording.
  • ~/enable_mapping: enable spatial mapping.
  • ~/enable_obj_det: enable object detection.
  • ~/toggle_svo_pause: set/reset SVO playing pause. Note: Only available if general.svo_realtime is false and if an SVO as been chosen as input source.

Services can be called using the rqt graphical tool enabling the plugin Plugins -> Services -> Service caller.

It is possible to call a service also using the CLI command $ ros2 service call.

For example to start Object Detection for the ZED2 camera:

$ ros2 service call /zed2/zed_node/enable_obj_det std_srvs/srv/SetBool data:\ true\

and the service server will reply:

waiting for service to become available...
requester: making request: std_srvs.srv.SetBool_Request(data=True)

response:
std_srvs.srv.SetBool_Response(success=True, message='Object Detection started')

Assigning a GPU to a camera

To improve performance, you can specify the gpu_id of the graphic card that will be used for the depth computation in the common.yaml configuration file. The default value (-1) will select the GPU with the highest number of CUDA cores. When using multiple ZEDs, you can assign each camera to a GPU to increase performance.

Limitations

Performance

This wrapper lets you quickly prototype applications and interface the ZED with other sensors and packages available in ROS. However, the ROS layer introduces significant latency and a performance hit. If performance is a major concern for your application, please consider using the ZED SDK library.