ZED SDK 3.0

What’s New

ZED SDK 3.0 is the first release that supports the brand new ZED 2 camera, improved in every way with wider angle optics, more accurate factory calibration, built-in new generation environmental and motion sensors and industrial grade mounting system and enclosure.

The SDK 3.0 also introduces new neural depth sensing, improved positional tracking, new AI-based object detection API that detects and tracks objects in 3D, image quality enhancement, performance improvements and a lot more features.

To update to 3.0 from previous 2.X versions, read the migration guide.

3.0.3

SDK

• Fixed an issue with recording and streaming with ZED cameras that could lead to a crash of NVENC encoder.
• Added support in 3.0 for ZED stereo cameras with firmware 1142 on Windows.
• Fixed Ubuntu16 installer missing turbojpeg dependency.
• Fixed Qt5 OpenGL/SVG dependencies in Ubuntu installers.

3.0.2

SDK

• Improved Spatial mapping performance by up to 200% on desktop and Jetson embedded platforms. Depth fusion is now much faster and the resulting model now refreshes in real-time on most platforms.
• Improved DEPTH_MODE::PERFORMANCE. It now benefits from neural depth estimation with improved accuracy, edges and cleaner data.
• Increased DEPTH_MODE::PERFORMANCE speed with a performance boost of up to 15%.
• Fixed a bug in resetTracking function. The yaw angle defined by the Transform parameter was not taken into account.
• Eliminated flickering in VIEW::DEPTH image when the scene has very low depth content.
• Fixed cuDNN reported version on JetPack 4.2. The model failed to run due to an incorrect reported version. It now works correctly.
• Removed libturbojpeg-0-dev dependency from Ubuntu 16 installer since the package is only available in Ubuntu 18.

3.0.1

No ZED SDK changes in this release (ZED Tools and integrations updates only).

3.0.0

New Features

Object Detection

• Added a new module that detects and tracks objects in 2D and 3D, compatible with ZED 2.
• Added enable/disableObjectDetection and retrieveObjects functions.
• Added ObjectData class that contains detected object data, such as 2D and 3D bounding box, tracking information, dimensions and 2D mask.

Depth

• Introduced neural networks for depth estimation in ULTRA mode to improve stereo depth accuracy. Other modes will be updated in following SDK updates.
• Introduced new confidence filter that detects homogenous zones. Use textureness_confidence_threshold to remove noisy values in the depth map such as sky or untextured areas.

Video

• Added image enhancement setting to provide more vivid and accurate colors. Enabled by default in 3.0.
• Added image sharpness setting to improve sharpening of live and recorded videos.
• Updated SVO video format to store new sensors data.

Sensors

• Introduced new Sensor API to provide access to data from IMU, barometer, magnetometer and temperature sensors in the ZED 2.

Local Streaming

• Improved local network streaming of video data between sending and receiving host computers.
• Video Streaming API is not compatible between 2.8 and 3.0.

Docker

• Improved Docker support for all ZED SDK versions.
• Introduced lightweight runtime Docker images for deployment of Docker applications using the ZED SDK.
• Updated StereoLabs DockerHub repository with the new images.

Performance

• Improved speed of the Depth module by carefully optimizing critical functions on desktop and embedded platforms.
• Optimized image rectification pipeline.

Software Support

• Updated to C++14 (previously C++11)
• The new Object Detection module uses TensorRT and cuDNN to run efficiently on embedded platforms. The module is dynamically loaded from a separate sl_ai library to avoid the explicit dependency when using sl_zed.

Platforms

• Added support for CUDA 10.2 and JetPack 4.3.
• Dropped support for JetPack 3.3 and Windows 7. While ZED may still be used on Windows 7, it is not officially supported.

Major 3.0 API Changes

Since there are a lot of changes compared to ZED SDK v2, a migration guide is available to help you switch easily from 2.8 to 3.0.

General

• Each enum is now an enum class (forced using TYPE::VALUE, instead of the current TYPE::TYPE_VALUE or even TYPE_VALUE).
• All deprecated functions/parameters/options have been removed.
• All libraries are now merged into sl_zed for clarity and ease of deployment.
• All headers are now merged into a single sl/Camera.hpp.
• Camera::grab is now blocking (guaranteed to return a new frame).

Functions

• SaveAs functions have been replaced by Mat::write().
• Added getter functions to retrieve any module parameter from Camera, such as getPositionalTrackingParameters.
• Added static function Camera::reboot() to perform a hardware reset of the camera.

Video

• Resolution and FPS information have been moved to Camera::getCameraInformation.
• The CAMERA_SETTINGS enum is renamed to VIDEO_SETTINGS.
• Added InitParameters::image_enhancement for better black level correction, AWB adjustment and improved gamma curve
• Added VIDEO_SETTINGS::SHARPNESS, a digital two-pass sharpening control that improves image quality.
• Video recording is now automatically handled by the grab function, without having to explicitly call the record() function (now removed).
• Added a new pauseRecording function.
• Added RecordingParameters structure.
• Deprecated RAW and LOSSY compression modes. They have been removed from SDK recording options.
• AVCHD and HEVC have been renamed into H264 and H265.

Sensors

• Added SensorsData class to store the data of all the other sensors present in the new ZED 2 which includes BarometerData, TemperatureData, MagnetometerData and IMUData. It also includes a state; CAMERA_MOTION_STATE which indicates if the camera is static, moving or falling.
• Camera::getIMUData() has been replaced by Camera::getSensorsData() to reflect the class update. It provides access to all the sensor values.

Depth

• Depth range and confidence parameters have been moved into InitParameters.
• Improved ULTRA mode with better accuracy and contours.

Positional Tracking

• Positional Tracking module function have been renamed to include “Positional” to differentiate it from object tracking. For instance: Camera::enableTracking is now Camera::enablePositionalTracking.
• Spatial memory has been renamed to Area memory for consistency with Area files (enable_area_memory, saveAreaMap)

Streaming

• Added StreamingParameters::chunk_size to select the maximum size of each transmitted packet and improve packet transmission.

Spatial Mapping

• Removed all deprecated Mesh functions. The overloaded SpatialMap functions that include support for both Mesh and FusedPointCloud should now be used.
• extractWholeSpatialMap is now smarter if a previously retrieved map is passed as an argument. It will retrieve only the difference between the initial and updated mesh.

Compatibility

• Streaming module is not compatible between 2.8 and 3.0.
• Due to a change in SVO metadata, SVOs recorded with 3.0 will only be compatible with SDK 2.8.4 and upwards.