ZED SDK 4.1

What’s New

This release of the ZED SDK is now a stable release of the major version 4. It brings significant stability and performance improvements. We’re introducing a new generation of the Positional Tracking module for more precise and robust localization. A new version of SVO is also available, recording full sensor data and supporting custom user data for ease of use.

4.1.4

Released: Sep 10, 2024

Bug Fixes

• Resolved issues with SVO recording and playback. Improved LOSSLESS modes performances and robustness to corruption while using less memory. SVO repair uses less memory. Resolved reading problems on Windows, as well as SVO real time mode.
• Added HDR mode support for ZED One 4K.

4.1.3

Released: Jul 16, 2024

SDK

• Added status reporting for NEURAL depth model optimization errors. The Camera::open() function can now fail to reflect the NEURAL depth status.
• Enabled mask retrieval for custom object detection.
• Improved GNSS calibration speed when covariance is very low, such as with RTK.

Bug Fixes

• Fixed an issue with the quad camera configuration that prevented all cameras from opening on JetPack 6.
• Fixed the depth min and max getter methods in the Python API.
• Fixed an issue with custom object detection where the 3D bounding box height increased along with the width.
• Fixed several edge cases in positional tracking generation 2.

4.1.2

Released: May 24, 2024

SDK

• Improved fusion sources synchronization for enhanced robustness in case of inconsistent data rates.
• Improved robustness of Positional Tracking Gen 2 IMU initialization.
• Added support for the JetPack 6.0 Production Release.

Bug Fixes

• Fixed a memory leak when closing the camera during SVO playback.
• Fixed a bug that could prevent recording lossless H26X SVO on Jetson.
• Fixed a regression in the image acquisition pipeline for HD1080 at 30FPS in USB mode that could impact performance.
• Fixed SVGA capture mode with ZED X cameras, which could result in black images.

4.1.1

Released: May 7, 2024

SDK

• Improved USB reliability on Linux platforms.

Bug Fixes

• Enhanced the reliability of ZED SDK streaming features, addressing issues where CAMERA_NOT_INITIALIZED errors occurred randomly.
• Corrected GNSS_RTK_FIX reporting errors that incorrectly displayed GNSS DATA NEED FIX.
• Fixed bad GNSS heading value.
• Fixed positional tracking gen2 potential random behavior after it relocates.
• Fixed a crash that could occur when reading a corrupted H264/H265 SVO2 on Jetson.
• Fixed a crash when reading/writing unsupported SVO files on Jetson Nano. The enableRecording function now returns an ERROR_CODE due to the absence of a hardware encoder on this platform. A software fallback will be introduced in a future release.
• Fixed SVO2 recording issues related to image-based compression or lossless H26X.

4.1.0

Released: Apr 3, 2024

New Features

Positional Tracking GEN2

• Introduced a new Positional Tracking generation, GEN2. It can be enabled using PositionalTrackingParameters::mode = POSITIONAL_TRACKING_MODE::GEN_2. It outputs more robust and precise trajectories compared to GEN1, leveraging IMU data at full frequency for better motion estimation. This version requires the new SVO2 data. A new warning ERROR_CODE::SENSORS_DATA_REQUIRED has been introduced to inform about missing high-frequency data when using the module. The state enum has been reworked for clarity on the module status; ODOMETRY_STATUS, SPATIAL_MEMORY_STATUS, and POSITIONAL_TRACKING_FUSION_STATUS are now available. It’s accessible using the new struct PositionalTrackingStatus.

Recording using SVO2

• Introduced a new default SVO2 format, recording sensor data at full frequency rate.
• Supports custom user data using new functions Camera::ingestDataIntoSVO alongside a new struct SVOData. The data can then be retrieved when opening the file using Camera::retrieveSVOData. Each data stream is associated with a string key, the data points are associated with a timestamp to replay synchronously if needed. Camera::getSVODataKeys retrieves all custom data stream keys present in the current SVO2 file. This format is mandatory to use the new Positional Tracking Gen2 with IMU data for best accuracy. The older format can still be recorded by setting the environment variable ZED_SDK_SVO_VERSION to 1.

Improved Neural Depth

• Introduced a new NEURAL_PLUS (NEURAL+) mode providing higher quality than Neural with sharper edges and more accurate depth, though requiring more computation.
• Improved NEURAL mode accuracy and confidence. This new model is more accurate with no runtime difference. The confidence has been reworked for accurate edges and minimized flying pixels. It also improves confidence estimation in challenging areas such as sky, hard illuminations, very close objects, etc., to remove false depth measurements reliably. Neural modes from previous versions can be accessed on demand.

Region of interest per module

• Reworked the Region of Interest to be applied independently for each module instead of implicitly for all at once. This feature allows masking an irrelevant part of the frame to avoid polluting results with unnecessary data. Typical usage includes masking a vehicle part where the camera is mounted to improve positional tracking precision and irrelevant depth data for obstacle detection. It can also be used to detect objects or body skeletons on only some part of the frame. The feature is now applied to each module through a new MODULE enum, with defaults applying to all modules. Functions Camera::setRegionOfInterest and Camera::getRegionOfInterest can be used to set the ROI. Auto-detection mode can find the static part of the image automatically by moving the camera for some time. It can be enabled using Camera::startRegionOfInterestAutoDetection, with RegionOfInterestParameters for adjusting detection parameters.

Global localization / Geo Tracking

• Improved Global Localization module accuracy and robustness, compatible with the new Positional Tracking GEN2. It now better behaves across a wider range of settings, especially across GNSS signal precision, from very precise RTK to dead reckoning.
• Supports and requires an antenna position relative to the camera, as well as the type of GNSS fix (2D, 3D, RTK FIX, etc.) to improve precision of the fusion. Samples have been improved for clarity, utilizing SVO2 custom data to record GNSS data, allowing a single recorded SVO2 file to contain all necessary data.

General

• Improved ZED X image quality by reducing motion blur and optimizing ISP to limit exposure time, resulting in better image quality and positional tracking precision improvements. Installing the SDK will automatically install the ISP config override for all ZED X cameras.
• Improved image validity check module on the ZED 2 and ZED 2i to detect rare cases of auto exposure getting stuck underexposing the left image.
• Improved ZED X capture process for more stable opening of multiple cameras and an optimized color conversion workflow.
• Reduced CPU usage and memory allocations in the ZED X capture process.
• Improved depth minimum range to better estimate working range based on camera baseline, lens, and depth algorithm used.
• Enhanced performance of Body Tracking models for better runtime performance and accuracy compared to version 4.0.
• Fixed a random issue where the depth minimum value was clamped at 1.5m.

Fusion

• Improved Body Tracking fusion synchronization process to better handle low or irregular framerates.
• Fixed the depth unit returned by retrieveMeasure, the user-defined UNIT was not applied.

Bug Fixes

• Fixed AI performance regression for NVIDIA Pascal GPUs (GTX 1070, GTX 1080, etc.), correcting incorrect fp16 execution without hardware support, leading to poor performance compared to fp32.
• Fixed grab recovery that lost camera video settings.
• Fixed a crash that could occur when calling the resetPositionalTracking function.
• Fixed a crash when using the Object detection masks that could get corrupted.