STEREOLABS

ZED SDK

3.6.1

What's New

2021-10-27

ZED SDK 3.6 introduces a major improvement in the Object detection module, custom detector inputs can be ingested into the SDK to have 3D detection and tracking from 2D bounding box. A new model that detects the person heads for improve accuracy in crowded environments. It now also supports ZED2/2i and ZED Mini.

This release adds support for the newer CUDA 11 minor versions, the CUDA 11 installer can be used for all CUDA 11 versions (11.0(*), 11.1, 11.2, 11.3, 11.4). TensorRT library for the AI module has been updated to 8, the ReID feature is now supported for all platforms. JetPack 4.6 is now supported.

Finally, It also adds new skeleton format, allowing the user to get the local keypoint orientation and position (relative to its parent's coordinate frame) when body fitting is enabled. This is useful to simplify avatar animation using software like Unity or Unreal for example.

3.6.1

Bug Fixes

  • Upgraded TensorRT version to 8.2 on Desktop to avoid potential issues when using CUDA 11.4 and the object detection module

3.6.0

New Features

API
  • Added function sl::Camera::ingestCustomBoxObjects to extract 3D data and perform tracking of custom 3D bounding boxes.

    • Added CustomBoxObjectData structure to ingest the external bounding box detections.
    • Samples using custom model with TensorRT or OpenCV DNN inference are now available.
  • Added a new detection mode sl::DETECTION_MODEL::PERSON_HEAD_BOX for person head detection and tracking.

    • It may be beneficial for application in crowded scene where persons in background are merely detected by the general purpose person detection model.
  • Added new skeleton format sl::BODY_FORMAT::POSE_32

    • The new format enable local keypoint orientation and position extracting when body fitting is enabled. The old format is always available with sl::BODY_FORMAT::POSE_18 and behaves exactly the same as previous ZED SDK version. sl::BODY_FORMAT ObjectDetectionParameters::body_format has been added to Object Detection init parameters to set the output skeleton format.
  • The ZED SDK now informs the user about invalid serial number detected for an MCU and suggest to run "ZED_Diagnostic -r" to recover the camera
  • Added missing imu/magnetometer, in sl::SensorsConfiguration::imu_magnetometer_transform transform for ZED2 and ZED2i
  • Added the new InitParameters::open_timeout_sec to set the maximum period of opening retry in case of connection issues
  • Improved open function
  • Added the fullReboot parameter to sl::Camera::reboot to select a full reboot or a simple UVC module reboot
Samples
  • Added samples using custom model input with TensorRT or OpenCV DNN inference.
Tools
  • Improved ZED Diagnostic:
    • can now recover a ZED Mini, ZED2 or ZED2i with invalid eeprom if started from the command line with option '-r'
    • can remove/clear all AI models files (engines and optim) with option '-aic'
    • can download all AI models with option '-aid'
    • can optimize (and download if need) all AI models with option '-aio'
    • can optimize (and download if need) a specific AI Model '- ais X' with X as sl::DETECTION_MODEL (int)
  • Sensor Viewer: Improved Magnetometer calibration process with more informations

Bug Fixes

API
  • Fixed `sl::Camera::reboot` command for ZED2i
  • Improved grab recovery behavior
  • Fix wrong values for camera/imu transform in translate coordinate
  • Object and Skeleton tracking are now automatically reset if the user sets a new SVO position that is far enough from the current SVO position. This allows for example to loop on a SVO.
Tools
  • Fixed ZED Diagnostic report information about MCU for ZED2i
  • MCU Firmware upgrade now works correctly also for Windows

Notes

  • JetPack 4.6 is now supported.
  • (*)The new CUDA 11 installer supports all minor versions (except CUDA 11.0 for Windows which is no longer supported)
  • CUDA 10.0 and JetPack 4.3 installers were deprecated and now no longer support AI module. They can still use all other SDK modules, users are encouraged to upgrade.
  • On Linux, since 3.5, the static version of the SDK doesn't include the sl_ai library anymore for performance and support reasons. `libsl_ai.so` is now needed to use the AI module even when using the static sl_zed library. On Jetson the static library is now available for all Jetpack versions. The installers are now unified again and include everything while being smaller in size.
  • The ReID feature (object batching) is now supported for all platforms
  • Dropped Ubuntu 16 support that was EOL

Integrations

LiveLink
  • Introduced a Live link plugin for Unreal Engine which allows to send skeleton tracking data to UE4 as well as an Unreal engine project already set up to receive this data and animate 3D models.
  • See our LiveLink repository.
C-API
  • Introduced a new C wrapper for creating C applications with the ZED SDK.
  • The C API is now available on GitHub.
Python
  • Updated documentation
  • Added support of ZED SDK 3.6
  • Added support of the new skeleton API
  • Added support of the new Head and Custom detection
  • Added access to IMU raw data (sl.IMUData.get_angular_velocity_uncalibrated and sl.IMUData.get_linear_acceleration_uncalibrated)
  • Improved the body tracking sample with a new display and better performance
  • Added access to IMU effective rate (sl.IMUData.effective_rate)
  • Removed dependency of CUDA in the whl files and simplified jetson versionning
C-sharp
  • Added support of ZED SDK 3.6
  • Added support of the new skeleton API
  • The C# API now uses the newly release C wrapper.
  • The dependencies (C wrapper and C# interface) are now regrouped into one nuget package to ease its use.
ROS
  • Fixed sensors topics publication for ZED2i
  • Fixed sensor_msgs type for depth image in OpenNI mode, from sensor_msgs::image_encodings::mono16 to sensor_msgs::image_encodings::TYPE_16UC1. Depth image in OpenNI mode is now compatible with the nodelet depthimage_to_laserscan
ROS2
  • Migration to ROS2 Foxy Fitzroy
  • Added xacro support for automatic URDF configuration
  • Reworked launch files to support xacro and launch parameters
    - Use ros2 launch zed_wrapper -s to retrieve all the available parameters
  • Added svo_path:='full path to SVO file' as input for all the launch files to start the node using an SVO as input without modifying common.yaml
  • Improved diagnostic information adding elaboration time on all the main tasks
  • Improved diagnostic time and frequencies calculation
  • Added StopWatch to sl_tools
  • Enabled Diagnostic status publishing
  • Changed the default values of the QoS parameter reliability for all topics from BEST_EFFORT to RELIABLE to guarantee compatibility with all ROS2 tools
  • Fixed tab error in zedm.yaml
  • Fixed compatibility issue with ZED SDK older than v3.5 - Thanks @PhilippPolterauer

Legacy

For older releases and changelog, see the ZED SDK release archive.

SDK Downloads

The ZED SDK allows you to add depth, motion sensing and spatial AI to your application. Available as a standalone installer, it includes applications, tools and sample projects with source code. Please check out our GitHub page and SDK documentation for additional resources.

Maintenance mode versions (legacy)

These versions are too old and no longer fully supported, the AI module is not available for these platforms.

Legacy

For older releases and changelog, see the ZED SDK release archive.

Integrations

Build applications with ZED and your favorite tools and languages using these integrations. View all integrations

App

ZED World is a standalone application that allows you to experience mixed reality with stereo pass-through in VR headsets. Requires ZED Mini, Oculus Rift or HTC Vive.