Bug Fixes

API

• Fixed a crash with CUDA 11.1 when loading sl::DETECTION_MODEL::HUMAN_BODY_* models on Linux during model optimization.
• Added security in sl::Camera::open() when serial number is detected as 0. This previously returned sl::ERROR_CODE::SUCCESS even when the serial number was invalid. This now returns sl::ERROR_CODE::CAMERA_NOT_DETECTED.

Bug Fixes

API

• Fixed dynamic load of nvidia-encoder library that could potentially break the encoding in docker environment (Linux only, introduced in 3.5.3).
• Fixed potential memory leak on streaming module (receiver side).

Tools

• Fixed ZED Explorer tool with --dc option when used for ZED 2i. It was reporting a wrong serial number.
• Fixed ZED Calibration tool with --cimu option for ZED 2 and ZED 2i. It now properly resets the runtime bias saved in memory.

Bug Fixes

API

• Fixed minor memory leaks on several modules (Object Detection, tracking, recording).
• Fixed a potential bug in LIVE mode when reading serial number (could lead to a buffer overflow).
• Fixed an issue on Normal map generation that was producing more outliers than expected. It is now fixed.
• Fixed wrong YUV4:2:2 to RGB conversion when using SVO real time mode.

Bug Fixes

API

• Fixed automatic recovery on ZED 2/2i when camera was falling in a over-exposed frozen state (on left and/or right camera). This was introduced in 3.5.1 and it was not performing correctly. It should now be really fixed.
• Fixed some thread sanitizer warning on Desktop versions reported by the ZED SDK.
• Added automatic power-cycle of the ZED 2/2i when not correctly detected. This could happen on Jetson platform when the camera was already plugged in before the Jetson was booting up. This is now automatically handled in the sl::Camera::open() function.
• Fixed SVO Decoding and Local Streaming on JetPack4.5 that was freezing with a NVBuffer transform error message. This bug was introduced in 3.5.1 during the decoding rework and it is now fixed.

Bug Fixes

API

• Fixed bug in SVO playback on Jetson when sl::ERROR_CODE::END_OF_SVOFILE_REACHED was not reached, leading to a infinite loop.
• Reworked SVO decoding (H264/5) on Jetson and Desktop leading to a more precise sl::Camera::setSVOPosition().
• Fixed a potential random crash in SVO decoding on Jetson when decoder was closed.
• Improved IMU data filtering for a more stable orientation given by the IMU. More precisely, IMU acceleration filtering has been increased to reduce its intrinsic noise.
• Fixed crash on sl::Camera::close() when using multiple cameras at the same time, especially when one camera was close before others.
• Fixed potential infinite freeze issue in sl::Camera::open() function when using streaming input mode.
• Added automatic recovery on ZED 2/2i when camera was falling in a over-exposed frozen state (on left and/or right camera). Note that this only works for ZED 2 and ZED 2i.
• Fixed sl::Camera::getSpatialMappingParameters() returning false values.
• Fixed object detection random crash (MULTI_CLASS_BOX_XX or HUMAN_BODY_XX) when the depth sl::SENSING_MODE::FILL parameter was used.

Tools

• Fixed ZED Sensor Viewer magnetometer calibration when using a ZED 2i
• Fixed ZED Explorer SpinBox for setting SVO position. It was not increasing/decreasing the current position.

Python

• Fixed inverted pixel coordinates in sl.Mat.get_value() function.

New Features

API

• Introduced new Body Tracking model with better performances and accuracy.
  • FAST mode is now x5(jetson)/x1.5(desktop) faster compared to FAST v3.4.
  • MEDIUM mode is x2(jetson) faster compared to FAST v3.4.
  • MEDIUM mode is 28% more precise compared to FAST v3.4
  • ACCURATE mode is now x2 faster compared to ACCURATE v3.4
  • ACCURATE mode is now 10% more precise compared to ACCURATE v3.4
  • This new model can also detect bodies in a much further range than previous version.
• Introduced new batching system for retrieving Objects in the Object Detection module.
  This adds a latency in the objects output flow, which allows internally a Re-Identification of the objects and check if an existing ID can be re-assigned.
  Therefore, an object that has been occluded/hidden or just moved out of the scene can now kept its original ID if the match is successful.
  
  • The objects can now be retrieved in batches of objects data defined by the new sl::ObjectsBatch structure.
  • Activate the batching system with sl::ObjectDetectionParameters::BatchParameters. See the API documentation for more information.
  • Call sl::Camera::retrieveObjects() to ingest the live objects into the batching system.
  • Retrieve batched objects with sl::Camera::getObjectsBatch(std::vector&).
  • A sl::ObjectsBatch is bound to a specific ID. it contains the list of timestamps, positions, bbox,... Each list have the exact same size.
• Updated Object detection models (MULTI_CLASS models) which slighty improve runtime (5%) and accuracy (2.5%).
• Added MULTI_CLASS_BOX_MEDIUM mode, x2 faster compared to ACCURATE mode while being 20% more precise compared to FAST.
• Added support of upcoming ZED 2i new camera (USB: 2b03:f880/f881).
• Improved sl::TRACKING_STATE::SEARCHING, in Object Detection module, leading to better recovery and avoid "flying" boxes.
• Improved Streaming stability on Windows.

Samples

• Updated object detection/birds eye viewer sample with a flag to activate/de-activate batch to show how to use the new batching system.
• Added a new Body Tracking Tutorial.

Tools

• Added documentation about Magnetometer Calibration using Sensor Viewer.

Bug Fixes

API

• Fixed a memory leak impacting the Object Detection module when using multi class box models.
• Fixed issue with magnetometer calibration parameters not correctly saved in camera internal memory. Note that this requires to perform a new magnetometer calibration (with ZED Sensor Viewer tool) as previous are invalid in the new system.
• Fixed non valid IMU data on some SVOs on last frame (skipped).
• Fixed the depth confidence value which could be higher than 100.

Tools

• Fixed ZEDfu crash on Jetson platform.
• Fixed the "Edges" rendering mode on ZEDfu for Jetson.
• Fixed Depth Viewer with multi screen and improve Settings Panel GUI.

Notes

• The AI module library is no longer integrated into the sl_zed static library to avoid performance and compatibility issues. To use the AI module in static compilation, the dynamic library of sl_ai will still be needed. JetPack 4.4 and 4.5 now have support for the static library
• The new batching system for object detection is only available with TensorRT 7.1. Since TensorRT 7.1 is not supported for all CUDA version, the module is only supported on the following platforms:
  • Windows with CUDA 11.0
  • Linux with CUDA 10.2 or 11.0
  • Jetson with JP44 and JP45
• This is the last ZED SDK release to support Ubuntu 16.04, since it reached end-of-life in April 2021.
• Fixed the "Edges" rendering mode on ZEDfu for Jetson.
• Fixed Depth Viewer with multi screen and improve Settings Panel GUI.

Integrations

Python

• Dropping Python 3.5 support (not maintained since September 2020)
• Added support of ZED SDK 3.5
• Added support of upcoming ZED 2i camera
• Added support of the new batching system
• Updated the object detection/bird view sample with a flag to activate/de-activate batch to show how to use the new batching system.
• Improved the body tracking sample with a new display and better performance
• Added a new Body Tracking Tutorial
• Fixed sl.Mat().set_value and sl.Mat().set_to functions.
• Fixed sl.Mat().get_data function for a Mat of type sl.MAT_TYPE.U16_C1.

GStreamer

• Depth mode has now a default value of NONE.
  To disable the depth elaboration it is also necessary to disable the positional tracking and the object detection, so the values of the relative activation options have been set to false as default.
  The zedsrc element now starts without performing any depth elaboration with the default settings.
• Added missing Positional Tracking parameters
• Added missing Runtime parameters (texture confidence, confidence, sensing mode and measure reference frame).
• Added missing Object Detection parameters (for body fitting and max range)
• Change parameters names in zedsrc to match the names in the ZED SDK

C#

• Added support of ZED SDK 3.5
• Added support of upcoming ZED 2i camera
• Added support of the new batching system from ZED SDK 3.5
• Updated the object detection/bird view sample with a flag to activate/de-activate the new batching system.
• Updated the body tracking sample with a new display
• Added a new Body Tracking Tutorial

OpenNI2

• Added online documentation

ROS

• Added support for new DEPTH16_MM data type for depth (OPENNI MODE)
• Fixed issue #660: detected objects topic not published if depth computation not active
• Improved support for ZED Object Detection
• Added Skeleton Tracking support
• New Rviz plugin for Object Detection in zed-ros-examples
• New parameters and name changing to fit the new OD features, also the start_object_detection service has been modified to match the new features:
  • new model parameter to choose the AI model
  • new max_range parameter to limit the detection range
  • new sk_body_fitting parameter to enable Skeleton fitting for skeleton AI models
  • people -> mc_people to indicate that it is related to multiclass AI models
  • vehicles -> mc_vehicles to indicate that it is related to multiclass AI models
  • new mc_bag parameter to enable bags detection with multiclass AI models
  • new mc_animal parameter to enable animals detection with multiclass AI models
  • new mc_electronics parameter to enable electronic devices detection with multiclass AI models
  • new mc_fruit_vegetable parameter to enable fruits and vegetables detection with multiclass AI models

Open Capture Driver

• Fixed issue with framerate
• Added AEC AGC ROI support