ZED SDK 3.5

What’s New

ZED SDK 3.5 introduces a major improvement in the Body Tracking module which is now up to 2.5x faster and 30% more accurate, and adds a new medium mode.

Besides, it also introduces a new way to retrieve detected objects in batches, powered by deep learning Re-Identification and appearance matching. This allows to keep the same ID for a detected object even if it has been occluded, hidden or out of the scene for some time.

Finally, the ZED SDK 3.5 adds support for the upcoming ZED 2i.

3.5.6

Bug Fixes

API

• Fixed regression in performance for skeleton tracking introduced in 3.5.X version.
• Fixed thread sanitizer issue reported in Camera self-calibration.
• Fixed some jumps in object detection tracking due to over-estimated disparity covariance.
• Fixed incorrect automatic recovery for ZED 2/2i performed internally in sl::Camera::open() when camera was not properly detected.
• Fixed potential random crash in Streaming on Windows, on receiver side.

3.5.5

Bug Fixes

API

• Fixed a crash with CUDA 11.1 when loading sl::DETECTION_MODEL::HUMAN_BODY_* models on Linux during model optimization.
• Added security in sl::Camera::open() when serial number is detected as 0. This previously returned sl::ERROR_CODE::SUCCESS even when the serial number was invalid. This now returns sl::ERROR_CODE::CAMERA_NOT_DETECTED.

3.5.4

Bug Fixes

API

• Fixed dynamic load of nvidia-encoder library that could potentially break the encoding in docker environment (Linux only, introduced in 3.5.3).
• Fixed potential memory leak on streaming module (receiver side).

3.5.3

Bug Fixes

API

• Fixed minor memory leaks on several modules (Object Detection, tracking, recording).
• Fixed a potential bug in LIVE mode when reading serial number (could lead to a buffer overflow).
• Fixed an issue on Normal map generation that was producing more outliers than expected. It is now fixed.
• Fixed wrong YUV4:2:2 to RGB conversion when using SVO real time mode.

3.5.2

Bug Fixes

API

• Fixed automatic recovery on ZED 2/2i when camera was falling in an over-exposed frozen state (on left and/or right camera). This was introduced in 3.5.1 and it was not performing correctly. It should now be really fixed.
• Fixed some thread sanitizer warning on Desktop versions reported by the ZED SDK.
• Added automatic power-cycle of the ZED 2/2i when not correctly detected. This could happen on Jetson platform when the camera was already plugged in before the Jetson was booting up. This is now automatically handled in the sl::Camera::open() function.
• Fixed SVO Decoding and Local Streaming on JetPack4.5 that was freezing with a NVBuffer transform error message. This bug was introduced in 3.5.1 during the decoding rework and it is now fixed.

3.5.1

Bug Fixes

API

• Fixed bug in SVO playback on Jetson when sl::ERROR_CODE::END_OF_SVOFILE_REACHED was not reached, leading to an infinite loop.
• Reworked SVO decoding (H264/5) on Jetson and Desktop leading to a more precise sl::Camera::setSVOPosition().
• Fixed a potential random crash in SVO decoding on Jetson when decoder was closed.
• Improved IMU data filtering for a more stable orientation given by the IMU. More precisely, IMU acceleration filtering has been increased to reduce its intrinsic noise.
• Fixed crash on sl::Camera::close() when using multiple cameras at the same time, especially when one camera was closed before others.
• Fixed potential infinite freeze issue in sl::Camera::open() function when using streaming input mode.
• Added automatic recovery on ZED 2/2i when camera was falling in an over-exposed frozen state (on left and/or right camera). Note that this only works for ZED 2 and ZED 2i.
• Fixed sl::Camera::getSpatialMappingParameters() returning false values.
• Fixed object detection random crash (MULTI_CLASS_BOX_XX or HUMAN_BODY_XX) when the depth sl::SENSING_MODE::FILL parameter was used.

3.5.0

New Features

API

• Introduced new Body Tracking model with better performances and accuracy.
  • FAST mode is now x5(jetson)/x1.5(desktop) faster compared to FAST v3.4.
  • MEDIUM mode is x2(jetson) faster compared to FAST v3.4.
  • MEDIUM mode is 28% more precise compared to FAST v3.4
  • ACCURATE mode is now x2 faster compared to ACCURATE v3.4
  • ACCURATE mode is now 10% more precise compared to ACCURATE v3.4
  • This new model can also detect bodies in a much further range than previous version.
• Introduced new batching system for retrieving Objects in the Object Detection module.
  This adds a latency in the objects output flow, which allows internally a Re-Identification of the objects and check if an existing ID can be re-assigned.
  Therefore, an object that has been occluded/hidden or just moved out of the scene can now keep its original ID if the match is successful.
  
  • The objects can now be retrieved in batches of objects data defined by the new sl::ObjectsBatch structure.
  • Activate the batching system with sl::ObjectDetectionParameters::BatchParameters. See the API documentation for more information.
  • Call sl::Camera::retrieveObjects() to ingest the live objects into the batching system.
  • Retrieve batched objects with sl::Camera::getObjectsBatch(std::vector&).
  • A sl::ObjectsBatch is bound to a specific ID. It contains the list of timestamps, positions, bbox,… Each list has the exact same size.
• Updated Object detection models (MULTI_CLASS models) which slightly improve runtime (5%) and accuracy (2.5%).
• Added MULTI_CLASS_BOX_MEDIUM mode, x2 faster compared to ACCURATE mode while being 20% more precise compared to FAST.
• Added support of upcoming ZED 2i new camera (USB: 2b03:f880/f881).
• Improved sl::TRACKING_STATE::SEARCHING, in Object Detection module, leading to better recovery and avoid “flying” boxes.
• Improved Streaming stability on Windows.

Bug Fixes

API

• Fixed a memory leak impacting the Object Detection module when using multi class box models.
• Fixed non valid IMU data on some SVOs on last frame (skipped).
• Fixed the depth confidence value which could be higher than 100.

Notes

• The AI module library is no longer integrated into the sl_zed static library to avoid performance and compatibility issues. To use the AI module in static compilation, the dynamic library of sl_ai will still be needed. JetPack 4.4 and 4.5 now have support for the static library.
• The new batching system for object detection is only available with TensorRT 7.1. Since TensorRT 7.1 is not supported for all CUDA version, the module is only supported on the following platforms:
  • Windows with CUDA 11.0
  • Linux with CUDA 10.2 or 11.0
  • Jetson with JP44 and JP45
• This is the last ZED SDK release to support Ubuntu 16.04, since it reached end-of-life in April 2021.