New Features

• Improved Neural depth runtime by up to 50%
  • Depth FPS now reaches 60FPS on platforms such as Jetson AGX Orin, and 30FPS for 2 cameras.
  • This speed-up is a combination of network optimization, pruning and improved usage of FP16 inference for supported configuration. It applied to all GPU.
• Added a new sl::Camera::setRegionOfInterest function.
  • It defines a region of interest with a mask to focus on for all the SDK, discarding other parts.
  • It can be useful to filter out known objects that are irrelevant, such as vehicle hoods or drone propellers that can degrade positional tracking or depth estimation by adding noises. Two new samples are now available.
• Added a new sl::ObjectDetectionParameters::allow_reduced_precision_inference parameter.
  • This parameter enables AI inference with quantization to improve runtime while allowing a slight decrease in accuracy. By default all AI models are running in fp16 (half) with no accuracy difference when the GPU supports (requires a Jetson or Volta GPU and newer) it and fallback to fp32 otherwise. This mode allows int8 inference that provides x4 the throughput on supported operations compared to fp32 and decreases the memory usage.
  • The accuracy loss should not exceed 1-2% on the compatible models. For now, only HUMAN_BODY_xxxx models support it.

Improvements

• Improved Depth Viewer rendering pipeline, especially on Jetson. It is now up to 50% faster.
• Improved sharpening behavior, the lowest setting now disables it. The processing has also been improved and no longer produces color artifacts.
• Added sl::PositionalTrackingParameters::set_gravity_as_origin parameter. It allows you to set the odometry world position using sensor data. For instance, if true (default), the initial_world_transform will be aligned with IMU gravity by keeping the user's yaw.
• Added sl::PositionalTrackingParameters::depth_min_range parameter to change the minimum range of the odometry. It may be useful if we want to remove some fixed objects in near range.
• Added sl::ObjectDetectionParameters::prediction_timeout_s parameter to tune the prediction time when an object disappears. When an object is not detected anymore, the SDK will predict its positions for a short period of time before its state switches to SEARCHING. This parameter prevents the jittering of the object state when there is a short misdetection
• Improved floor plane detection stability
• Introduced a new sl::POSITIONAL_TRACKING_STATE::SEARCHING_FLOOR_PLANE, when you use the sl::PositionalTrackingParameter::set_floor_as_origin, to know if the positional tracking is started or not
• sl::Mat class has been improved to allow faster transfers. The updateGPUfromCPU and updateCPUfromGPU, as well as copyTo, now include a cudaStream_t parameter to asynchronously retrieve the data when using GPU memory.
• Added a support function, sl::convertImage, to convert image format from sl::MAT_TYPE::U8C4 to sl::MAT_TYPE::S8C4, mainly for Unreal Engine.
• On Linux, the installers now use zstandard as the default compression (previously gzip). They are now half the size and twice faster to extract. The zstd package must be installed (sudo apt install zstd) to use the ZED SDK installer.

Bug Fixes

• Fixed a slight drift in sl::Pose from the sl::Camera::getPosition function when using sl::REFERENCE_FRAME::CAMERA. The cumulative pose from this local pose could introduce a huge drift compared to the sl::Pose from sl::REFERENCE_FRAME::WORLD when loop closure is disabled

Integrations

Python

• Improve installation script with better python package folder detection (reported by GitHub user @GER-Jani).

Unreal

• Release of a new plugin compatible with Unreal Engine 5

Unity

• Add the option to open the camera by its serial number.

ROS 2

• Added support for ROS 2 Humble Hawksbill. Thx @nakai-omer. The two ROS2 LTS releases are now supported simultaneously.
• Fixed a wrong behavior of the set_pose service. Now the initial odometry is coherent with the new starting point value used as a parameter.
• Added Plane Detection feature. Read more.
• Added a sub-set of parameters for debugging with new parameters debug_mode and debug_sensors.
• Fixed the "NO DEPTH" mode. By setting depth.quality to 0 now the depth extraction and all the sub-modules depending on it are correctly disabled.
• Set read_only flag in the parameter descriptor for non-dynamic parameters. Thx @bjsowa.
• Enabled Intra Process Communication (IPC). The ZED node no longer publishes topics with TRANSIENT LOCAL durability.
• Added TF broadcasting diagnostic.
• Fixed data grabbing frame rate when publishing is set to a lower value.
• Improved IMU/Left Camera TF broadcasting at IMU frequency.
• Improved TF broadcasting at grabbing frequency.
• Fixed random errors when closing the node.
• New data thread configuration to maximize data publishing frequency.
• Sensor data publishing moved from timer to thread.
• RGB/Depth data publishing moved from timer to thread.
• Moved Sensor Parameters from cameras configuration files to common.yaml.
• Moved Object Detection parameters from cameras configuration files to common.yaml.
• The parameter general.sdk_verbose is now an integer accepting different SDK verbose levels.
• Added a new benchmarking tool in the zed-ros2-examples repository to get frequency and bandwidth information from a subscribed topic and plot data.

ROS

• Fixed TF timestamp issue in SVO mode.
• Added Plane Detection. Read more.
• Fixed wrong TF broadcasting when calling the set_pose, reset_tracking, and reset_odometry services. Now the initial odometry is coherent with the new starting point.
• Added parameter sensors/max_pub_rate to set the maximum publishing frequency of sensors data
• Improved Sensors thread.
• Fixed the frame links of barometer, magnetometer, and temperature sensors for ZED 2i.

Known issue

• Maxwell and Kepler GPUs (compute capabilities 35, 50 and 52) are not supported using CUDA 11.X, only with CUDA 10.2

Deprecation

• CUDA 10.0 installers are no longer supported with this release. Users are encouraged to migrate to the latest CUDA 11.X.
• While CUDA 10.2 is still supported, users are strongly encouraged to migrate to CUDA 11.X, which provides a performance improvement between 20 to 40% with the same hardware.
• Jetson users are encouraged to move the latest L4T 35.1 when using a Xavier and Orin, and L4T 32.7 for Nano and TX2. The L4T 32.3, 32.4, 32.5, 32.6 are now deprecated.