STEREOLABS

ZED SDK

3.5.0

What's New

2021-04-20

ZED SDK 3.5 introduces a major improvement in the Body Tracking module which is now up to 2.5 faster and 30% more accurate and a new medium mode.

Besides, it also introduces a new way to retrieve detected objects in batches, powered by deep learning Re-Identification and appearance matching. This allows to keep the same ID for a detected object even if it has been occluded, hidden or out of the scene for some time.

Finally, the ZED SDK 3.5 adds the support of the upcoming ZED 2i.

3.5.0

New Features

API
  • Introduced new Body Tracking model with better performances and accuracy.

    • FAST mode is now x5(jetson)/x1.5(desktop) faster compared to FAST v3.4.
    • MEDIUM mode is x2(jetson) faster compared to FAST v3.4.
    • MEDIUM mode is 28% more precise compared to FAST v3.4
    • ACCURATE mode is now x2 faster compared to ACCURATE v3.4
    • ACCURATE mode is now 10% more precise compared to ACCURATE v3.4
    • This new model can also detect bodies in a much further range than previous version.
  • Introduced new batching system for retrieving Objects in the Object Detection module.

    • The objects can now be retrieved in batches of objects data defined by the new sl::ObjectsBatch structure.

    • This adds a latency in the objects output flow, which allows internally a Re-Identification of the objects and check if an existing ID can be re-assigned.
      Therefore, an oject that has been occluded/hidden or just moved out of the scene can now kept its original ID if the match is successfull.

    • Activate the batching system with sl::ObjectDetectionParameters::BatchParameters. See the API documentation for more information.
    • Call sl::Camera::retrieveObjects() to ingest the live objects into the batching system.
    • Retrieve batched objects with sl::Camera::getObjectsBatch(std::vector&).
    • A sl::ObjectsBatch is bound to a specific ID. it contains the list of timestamps, positions, bbox,.... Each list have the exact same size.
  • Updated Object detection models (MULTI_CLASS models) which slighty improve runtime (5%) and accuracy (2.5%).
  • Added MULTI_CLASS_BOX_MEDIUM mode, x2 faster compared to ACCURATE mode while being 20% more precise compared to FAST.
  • Added support of upcoming ZED2i new camera (USB : 2b03:f880/f881).
  • Improved TRACKING_STATE::SEARCHING, in Object Detection module, leading to better recovery and avoid "flying" boxes.
  • Improved Streaming stability on Windows.
Samples
  • Updated object detection/birds eye viewer sample with a flag to activate/de-activate batch to show how to use the new batching system.
  • Added a new Body Tracking Tutorial.
Tools
  • Added documentation about Magnetometer Calibration using Sensor Viewer.

Bug Fixes

API
  • Fixed a memory leak impacting the Object Detection module when using multi class box models.
  • Fixed issue with magnetometer calibration parameters not correctly saved in camera internal memory. Note that this requires to perform a new magnetometer calibration (with ZED Sensor Viewer tool) as previous are invalid in the new system.
  • Fixed non valid IMU data on some SVOs on last frame (skipped).
  • Fixed the depth confidence value which could be higher than 100.
Tools
  • Fixed ZEDfu crash on Jetson platform.
  • Fixed the "Edges" rendering mode on ZEDfu for Jetson.
  • Fixed Depth Viewer with multi screen and improve Settings Panel GUI.

Notes

  • The AI module library is no longer integrated into the sl_zed static library to avoid performance and compatibility issues. To use the AI module in static compilation, the dynamic library of sl_ai will still be needed. JetPack 4.4 and 4.5 now have support for the static library
  • The new batching system for object detection is only available with TensorRT 7.1. Since TensorRT 7.1 is not supported for all CUDA version, the module is only supported on the following platforms :

  • - Windows with CUDA 11.0.
    - Linux with CUDA 10.2 or cuda 11.0.
    - Jetsons with JP44 and JP45.

  • This is the last ZED SDK release to support Ubuntu 16.04, since it reached end-of-life in April 2021.
  • Fixed the "Edges" rendering mode on ZEDfu for Jetson.
  • Fixed Depth Viewer with multi screen and improve Settings Panel GUI.

Integrations

Python
  • Dropping python 3.5 support (not maintained since September 2020)
  • Added support of ZED SDK 3.5
  • Added support of upcoming ZED2i camera
  • Added support of the new batching system
  • Updated the object detection/bird view sample with a flag to activate/de-activate batch to show how to use the new batching system.
  • Improved the body tracking sample with a new display and better performance
  • Added a new Body Tracking Tutorial
  • Fixed sl.Mat().set_value and sl.Mat().set_to functions.
  • Fixed sl.Mat().get_data function for a Mat of type sl.MAT_TYPE.U16_C1`.
GStreamer
  • Depth mode has now a default value of NONE.
    To disable the depth elaboration it is also necessary to disable the positional tracking and the object detection, so the values of the relative activation options have been set to false as default.
    The zedsrc element now starts without performing any depth elaboration with the default settings.
  • Added missing Positional Tracking parameters
  • Added missing Runtime parameters (texture confidence, confidence, sensing mode and measure reference frame).
  • Added missing Object Detection parameters (for body fitting and max range)
  • Change parameters names in `zedsrc` to match the names in the ZED SDK
C-sharp
  • Added support of ZED SDK 3.5
  • Added support of upcoming ZED2i camera
  • Added support of the new batching system from ZED SDK 3.5
  • Updated the object detection/bird view sample with a flag to activate/de-activate the new batching system.
  • Updated the body tracking sample with a new display
  • Added a new Body Tracking Tutorial
OpenNI2
ROS
  • Added support for new DEPTH16_MM data type for depth (OPENNI MODE)
  • Fixed issue #660: detected objects topic not published if depth computation not active
  • Improved support for ZED Object Detection
  • Added Skeleton Tracking support
  • New Rviz plugin for Object Detection in zed-ros-examples
  • New parameters and name changing to fit the new OD features, also the start_object_detection service has been modified to match the new features:
    - new model parameter to choose the AI model
    - new max_range parameter to limit the detection range
    - new sk_body_fitting parameter to enable Skeleton fitting for skeleton AI models
    - people -> mc_people to indicate that it is related to multiclass AI models
    - vehicles -> mc_vehicles to indicate that it is related to multiclass AI models
    - new mc_bag parameter to enable bags detection with multiclass AI models
    - new mc_animal parameter to enable animals detection with multiclass AI models
    - new mc_electronics parameter to enable electronic devices detection with multiclass AI models
    - new mc_fruit_vegetable parameter to enable fruits and vegetables detection with multiclass AI models
Open Capture Driver
  • Fixed issue with framerate
  • Added AEC AGC ROI support

Legacy

For older releases and changelog, see the ZED SDK release archive.

SDK Downloads

The ZED SDK allows you to add depth, motion sensing and spatial AI to your application. Available as a standalone installer, it includes applications, tools and sample projects with source code. Please check out our GitHub page and SDK documentation for additional resources.

Maintenance mode versions (legacy)

These versions are too old and no longer fully supported, the AI module is running older version of models and performance and accuracy can be significantly lower.

Legacy

For older releases and changelog, see the ZED SDK release archive.

Integrations

Build applications with ZED and your favorite tools and languages using these integrations. View all integrations

App

ZED World is a standalone application that allows you to experience mixed reality with stereo pass-through in VR headsets. Requires ZED Mini, Oculus Rift or HTC Vive.