Improvements

• Improved ZED, ZED Mini and ZED 2i connection stability on Linux and Nvidia Jetson by adding USB autosuspend udev rules

Bug Fixes

• Fixed the installation process of CUDA in the Linux installer for CUDA 11
• Fixed a crash when using corrupted SVOs as input

Integrations

Unreal Engine / Livelink

• Improved skeleton tracking integration.
• Fixed avatars height relative to the ground floor.
• Fixed floor texture tiling.
• Reduce skeletons blinking by displaying only skeletons with OK tracking states.

Unity

• Improved skeleton tracking integration.
• Fixed a bug where some avatars were not correctly animated.
• Fixed avatars height relative to the ground floor.
• Reduce skeletons blinking by displaying only skeletons with OK tracking states.

Improvements

• Adding support for Python 3.9 and 3.10.
• ZED Diagnostic correctly reports used and required CUDA version.
• Improved streaming receiver on Windows to have a smoother experience.
• Improved robustness of the AI model download process.
• Improved depth stabilizer, removed a ghost effect on dynamic edges.
• Adding a new scripts to help identify Linux platform versions (such as CUDA, Ubuntu and JetPack version) to select compatible ZED SDK installer.
  • Usage example:
    $ bash -c "$(curl -fsSL https://download.stereolabs.com/getzed)"

Bug Fixes

• Fixed various memory leak in tools.
• Fixed random GPU memory corruption leading to non repeatable output.
• Fixed an wrong behavior in sl::Camera::getAreaExportState() always returning success, now it returns the real state of the export process.
• Fixed a bug with AI model deflate where corrupted file could create an allocation issue and segmentation fault.
• Fixed an issue in ZED SVO Editor leading to a crash if the destination directory is not created.
• Fixed missing magnetometer values (heading, state, uncalibrated) in SVO or streaming input type.
• Fixed setSVOPosition(0) that could lead to wrong position in SVO file.

Integrations

Python

• Fixed an issue leading to uncomplete array when retrieving depth as DEPTH_U16_MM.
• Adding custom object detection API Python sample using PyTorch (YOLOv5)

Improvements

• Added CUDA 11.5 support.
• Added Windows 11 support.
• AI models download has been improved to be more robust against CDN issues (auto-retry download with alternative URL).
• AI models and optimized models are now pre-processed in RAM instead of temporary files, this makes the process faster and more robust especially with concurrent launches.
• Installer script on Linux now check first if the packages are already installed and don't require an internet connection if not needed.
• Improved Linux USB camera acquisition.
• Reduced stream latency.
• Improved stream runtime which decrease the number of dropped frames.

Bug Fixes

• Fixed an issue in Linux platform (Desktop and Jetson), leading to an important CPU load when using SDK as stream receiver.
• Fixed sl::DEPTH_MODE::ULTRA repeatability (without depth stabilization).
• Fixed ZED Calibration tool on large size screen (and improved stability).
• Fixed a memory issue when optimizing all AI models from ZED Diagnostic tool (option -aio).
• Added missing toString function for sl::BODY_FORMAT.
• Added missing packages libpng and libgomp from docker images

Bug Fixes

• Upgraded TensorRT version to 8.2 on Desktop to avoid potential issues when using CUDA 11.4 and the object detection module

New Features

API

• Added function sl::Camera::ingestCustomBoxObjects to extract 3D data and perform tracking of custom 3D bounding boxes.
  • Added sl::CustomBoxObjectData structure to ingest the external bounding box detections.
  • Samples using custom model with TensorRT or OpenCV DNN inference are now available.
• Added a new detection mode sl::DETECTION_MODEL::PERSON_HEAD_BOX for person head detection and tracking.
  • It may be beneficial for application in crowded scene where persons in background are merely detected by the general purpose person detection model.
• Added new skeleton format sl::BODY_FORMAT::POSE_34
  • The new format enable local keypoint orientation and position extracting when body fitting is enabled. The old format is always available with sl::BODY_FORMAT::POSE_18 and behaves exactly the same as previous ZED SDK version. sl::BODY_FORMAT ObjectDetectionParameters::body_format has been added to Object Detection init parameters to set the output skeleton format.
• The ZED SDK now informs the user about invalid serial number detected for an MCU and suggest to run ZED_Diagnostic -r to recover the camera
• Added missing imu/magnetometer, in sl::SensorsConfiguration::imu_magnetometer_transform transform for ZED 2 and ZED 2i
• Added the new sl::InitParameters::open_timeout_sec to set the maximum period of opening retry in case of connection issues
• Improved open function
• Added the fullReboot parameter to sl::Camera::reboot to select a full reboot or a simple UVC module reboot

Samples

• Added samples using custom model input with TensorRT or OpenCV DNN inference.

Tools

• Improved ZED Diagnostic:
  • can now recover a ZED Mini, ZED 2 or ZED 2i with invalid eeprom if started from the command line with option -r
  • can remove/clear all AI models files (engines and optimi) with option -aic
  • can download all AI models with option -aid
  • can optimize (and download if need) all AI models with option -aio
  • can optimize (and download if need) a specific AI Model - ais X with X as sl::DETECTION_MODEL (int)
• Sensor Viewer: Improved Magnetometer calibration process with more information

Bug Fixes

API

• Fixed sl::Camera::reboot command for ZED 2i
• Improved grab recovery behavior
• Fix wrong values for camera/IMU transform in translate coordinate
• Object and Skeleton tracking are now automatically reset if the user sets a new SVO position that is far enough from the current SVO position. This allows for example to loop on a SVO.

Tools

• Fixed ZED Diagnostic report information about MCU for ZED2i
• MCU Firmware upgrade now works correctly also for Windows

Notes

• JetPack 4.6 is now supported.
• (*)The new CUDA 11 installer supports all minor versions (except CUDA 11.0 for Windows which is no longer supported)
• CUDA 10.0 and JetPack 4.3 installers were deprecated and now no longer support AI module. They can still use all other SDK modules, users are encouraged to upgrade.
• On Linux, since 3.5, the static version of the SDK doesn't include the sl_ai library anymore for performance and support reasons. libsl_ai.so is now needed to use the AI module even when using the static sl_zed library. On Jetson the static library is now available for all JetPack versions. The installers are now unified again and include everything while being smaller in size.
• The ReID feature (object batching) is now supported for all platforms
• Dropped Ubuntu 16 support that was EOL

Integrations

LiveLink

• Introduced a Live link plugin for Unreal Engine which allows to send skeleton tracking data to UE4 as well as an Unreal engine project already set up to receive this data and animate 3D models.
• See our LiveLink repository.

C API

• Introduced a new C wrapper for creating C applications with the ZED SDK.
• The C API is now available on GitHub.

Python

• Updated documentation
• Added support of ZED SDK 3.6
• Added support of the new skeleton API
• Added support of the new Head and Custom detection
• Added access to IMU raw data (sl.IMUData.get_angular_velocity_uncalibrated and sl.IMUData.get_linear_acceleration_uncalibrated)
• Improved the body tracking sample with a new display and better performance
• Added access to IMU effective rate (sl.IMUData.effective_rate)
• Removed dependency of CUDA in the whl files and simplified Jetson versionning

C#

• Added support of ZED SDK 3.6
• Added support of the new skeleton API
• The C# API now uses the newly release C wrapper.
• The dependencies (C wrapper and C# interface) are now regrouped into one nuget package to ease its use.

ROS

• Fixed sensors topics publication for ZED 2i
• Fixed sensor_msgs type for depth image in OpenNI mode, from sensor_msgs::image_encodings::mono16 to sensor_msgs::image_encodings::TYPE_16UC1. Depth image in OpenNI mode is now compatible with the nodelet depthimage_to_laserscan

ROS2

• Migration to ROS 2 Foxy Fitzroy
• Added xacro support for automatic URDF configuration
• Reworked launch files to support xacro and launch parameters
  - Use ros2 launch zed_wrapper -s to retrieve all the available parameters
• Added svo_path:='full path to SVO file' as input for all the launch files to start the node using an SVO as input without modifying common.yaml
• Improved diagnostic information adding elaboration time on all the main tasks
• Improved diagnostic time and frequencies calculation
• Added StopWatch to sl_tools
• Enabled Diagnostic status publishing
• Changed the default values of the QoS parameter reliability for all topics from BEST_EFFORT to RELIABLE to guarantee compatibility with all ROS 2 tools
• Fixed tab error in zedm.yaml
• Fixed compatibility issue with ZED SDK older than v3.5 - Thanks @PhilippPolterauer