Getting Started with ROS

The ZED ROS wrapper lets you use the ZED stereo cameras with ROS. It provides access to the following data:

  • Left and right rectified/unrectified images
  • Depth map
  • Colored 3D point cloud
  • Visual odometry: Position and orientation of the camera
  • Pose tracking: Position and orientation of the camera fixed and fused with IMU data (ZED Mini only)

Installation

Prerequisites

Note: if you are using a ZED Mini and want to visualize the IMU information using RVIZ, you will also need to install the RVIZ IMU plugin:

 $ sudo apt install ros-kinetic-rviz-imu-plugin

Build the package

zed_ros_wrapper is a catkin package. It depends on the following ROS packages:

  • tf2_ros
  • tf2_geometry_msgs
  • nav_msgs
  • roscpp
  • rosconsole
  • sensor_msgs
  • stereo_msgs
  • image_transport
  • dynamic_reconfigure
  • nodelet

Note: If you haven’t set up your catkin workspace yet, please follow this short tutorial.

To install zed_ros_wrapper, open a bash terminal, clone the package from Github and build it:

$ cd ~/catkin_ws/src/ #use your current catkin folder
$ git clone https://github.com/stereolabs/zed-ros-wrapper.git
$ cd ..
$ catkin_make -DCMAKE_BUILD_TYPE=Release
$ echo source $(pwd)/devel/setup.bash >> ~/.bashrc
$ source ~/.bashrc

Note: If you are using a different console interface like zsh, you have to change the source command as follows: echo source $(pwd)/devel/setup.zsh >> ~/.zshrc and source ~/.zshrc.

Error: If an error mentioning /usr/lib/x86_64-linux-gnu/libEGL.so blocks compilation, use the following command to repair the libEGl symlink before restarting the catkin_make command:

#Only on libEGL error
$ sudo rm /usr/lib/x86_64-linux-gnu/libEGL.so; sudo ln /usr/lib/x86_64-linux-gnu/libEGL.so.1 /usr/lib/x86_64-linux-gnu/libEGL.so

Starting the ZED node

The ZED is available in ROS as a node that publishes its data to topics. You can read the full list of available topics here.

Open a terminal and use roslaunch to start the ZED node:

$ roslaunch zed_wrapper zed.launch

Note: You can set your own configuration parameters in the zed_camera.launch file available here.

Displaying ZED data

Using RVIZ

RVIZ is a useful visualization tool in ROS. Using RVIZ, you can visualize the ZED left and right images, depth, point cloud, and 3D trajectory.

Launch the ZED wrapper along with RVIZ using the following command:

$ roslaunch zed_display_rviz display.launch

If you are using a ZED Mini camera, you can visualize additional information about IMU data using the following command:

$ roslaunch zed_display_rviz display_zedm.launch

Note: If you haven’t yet configured your own RVIZ interface, you can find a detailed tutorial here.

Displaying Images

The ZED node publishes both original and stereo rectified (aligned) left and right images. In RVIZ, select a topic you and use the image preview mode. Here the list of the available image topics:

  • zed/rgb/image_rect_color: Color rectified image (left image by default)
  • zed/rgb/image_raw_color: Color unrectified image (left image by default)
  • zed/rgb/camera_info: Left camera calibration data
  • zed/right/image_rect_color: Color rectified right image
  • zed/right/image_raw_color: Color unrectified right image
  • zed/right/camera_info: Right sensor calibration data
  • zed/confidence/confidence_image: Confidence map as image

Note: The Confidence Map is also available as a 32bit floating point image subscribing to the /zed/confidence/confidence_map topic.

Displaying Depth

The depth map can be displayed in RVIZ with the following topic:

  • zed/depth/depth_registered: 32-bit depth values in meters. RVIZ will normalize the depth map on 8-bit and display it as a grayscale depth image.

Note: An OpenNI compatibility mode is available in the launch/zed_camera.launch file. Set openni_depth_mode to 1 to get depth in millimeters and in 16-bit precision, and restart the ZED node.

Displaying Disparity

The Disparity Image is available by subscribing to the zed/disparity/disparity_image topics.

Launch the Disparity Viewer to visualize it:

$ rosrun image_view disparity_view image:=/zed/disparity/disparity_image

Displaying the Point cloud

A 3D colored point cloud can be displayed in RVIZ with the zed/point_cloud/cloud_registered topic.

Add it in RVIZ with point_cloud -> cloud -> PointCloud2. Note that displaying point clouds slows down RVIZ, so open a new instance if you want to display other topics.

Displaying position and path

The ZED position and orientation in space over time is published to the following topics:

  • zed/odom: Odometry pose referred to odometry frame (only visual odometry is applied for ZED, visual-inertial for ZED Mini)
  • zed/pose: Camera pose referred to Map frame (complete data fusion algorithm is applied)
  • zed/pose_with_covariance: Camera pose referred to Map frame with covariance (if spatial_memory is false in launch parameters)
  • zed/path_odom: The sequence of camera odometry poses in Map frame
  • zed/path_map: The sequence of camera poses in Map frame

Important: By default, RVIZ does not display odometry data correctly. Open the newly created Odometry object in the left list, and set Position Tolerance and Angle Tolerance to 0, and Keep to1.

Launching with recorded SVO video

With the ZED, you can record and play back stereo video using the .svo file format. To record a sequence, open the ZED Explorer app and click on the REC button.

To launch the ROS wrapper with an SVO file, set an svo_file path launch parameter in the command line when starting the package:

roslaunch zed_wrapper zed.launch svo_file:=/path/to/file.svo

Important: Use only full paths to the SVO file. Relative paths are not allowed.

Dynamic reconfigure

You can dynamically change many configuration parameters during the execution of the ZED node:

  • confidence threshold: Sets a threshold that filters the values of the depth or the point cloud. With a confidence threshold set to 100, all depth values will be written in the depth and the point cloud. This is set to 80 by default, which removes the least accurate values.
  • auto_exposure: Enables/disables automatic gain and exposure
  • exposure: Sets camera exposure only if auto_exposure is false
  • gain: Set camera gain only if auto_exposure is false
  • mat_resize_factor: Sets the scale factor of the output images and depth map. Note that the camera will acquire data at the dimension set by the resolution parameter; images are resized before being sent to the user
  • max_depth: Sets the maximum depth range

You can set the parameters using the command dynparam set, e.g.:

$ rosrun dynamic_reconfigure dynparam set /zed/zed_wrapper_node confidence 80

…or you can use the GUI provided by the rqt stack:

$ rosrun rqt_reconfigure rqt_reconfigure