Sensing Modes #
The ZED SDK provides two modes for depth sensing:
Standard Mode #
STANDARD mode is the default depth sensing mode with the ZED. The
STANDARD mode preserves distance metrics and shapes and runs faster than the
FILL mode, but it contains holes due to visual occlusions and filtering. These holes are represented by black areas in the depth image. Use the
STANDARD mode for applications such as autonomous navigation, obstacle detection, 3D mapping, people detection and tracking.
STANDARD mode, holes (black pixels) have different values depending on their type. They are associated with an enum:
NANreferred as OCCLUSION_VALUE. The depth of the pixel cannot be estimated as it is occluded or an outlier.
-INFINITYreferred as TOO_CLOSE. The depth of the pixel cannot be estimated as it is too close to the camera.
INFINITYreferred as TOO_FAR. The depth of the pixel cannot be estimated as it is too far from the camera.
You can check for valid depth values using
Fill Mode #
FILL mode provides a fully dense depth map with a Z value for every pixel (X, Y) in the left image. The
FILL mode fills holes and occlusions in the depth map and adds a filtering stage that improves edges and temporal stability but can alter the actual distance of objects in the scene.
This mode is recommended for applications such as mixed-reality capture and visual effects. The
FILL mode requires more resources and runs at a lower FPS than the
To configure depth sensing in
FILL mode, set
// Set sensing mode in FILL RuntimeParameters runtime_parameters; runtime_parameters.sensing_mode = SENSING_MODE::FILL;
# Set sensing mode in FILL runtime_parameters =sl.RuntimeParameters() runtime_parameters.sensing_mode = sl.SENSING_MODE.FILL
// Set sensing mode in FILL RuntimeParameters runtime_parameters = new RuntimeParameters(); runtime_parameters.sensingMode = SENSING_MODE.FILL;
Depth Modes #
Several depth modes are available to fit your application’s needs. These settings adjust the level of accuracy, range and computational performance of the depth sensing module. They are listed ranked below from the better one to the faster one.
NEURAL: brings the depth sensing to the next level of accuracy by using AI techniques. Accurate and smooth, even in the most challenging situations.
ULTRA: offers the highest depth range and better preserved Z-accuracy along the sensing range for computer vision-based technique.
QUALITY: has a strong filtering stage giving smooth surfaces.
PERFORMANCE: designed to be smooth, can miss some details.
Generally speaking, we recommend using the
ULTRA mode for both desktop and embedded applications. If your application requires a lot of resources, switch to
// Set depth mode in ULTRA InitParameters init_parameters; init_parameters.depth_mode = DEPTH_MODE::ULTRA;
# Set depth mode in ULTRA init_parameters = sl.InitParameters() init_parameters.depth_mode = sl.DEPTH_MODE.ULTRA
// Set depth mode in ULTRA InitParameters init_parameters = new InitParameters(); init_parameters.depthMode = DEPTH_MODE.ULTRA;
Depth Range #
Depth range corresponds to the minimum and maximum distance at which the depth of an object can be estimated.
|ZED X||ZED X Mini||ZED 2i||ZED Mini|
|Default range||0.5m to 20m (1.6 to 65ft)||0.2m to 10m (0.7 to 33ft)||0.5m to 20m (1.6 to 65ft)||1.5m to 35m (4.9 to 115ft)||0.2m to 10m (0.7 to 33ft)|
|Min range||0.3m (1ft)||0.1m (0.3ft)||0.3m (1ft)||1.5m (4.9ft)||0.1m (0.3ft)|
|Max range||20m* (65ft)||8m* (26ft)||20m* (65ft)||35m* (115ft)||15m* (49ft)|
(*): The max range can be set to higher values (see Maximum Range below), but the depth’s accuracy might decrease significantly.
Minimum Range #
The minimum range can be lowered from its default value using the
depth_minimum_distance setting in
InitParameters init_parameters; init_parameters.coordinate_units = UNIT::METERS; init_parameters.depth_minimum_distance = 0.15 ; // Set the minimum depth perception distance to 15cm
init_params = sl.InitParameters() init_parameters.coordinate_units = sl.UNIT.METER init_parameters.depth_minimum_distance = 0.15 # Set the minimum depth perception distance to 15cm
InitParameters init_parameters = new InitParameters(); init_parameters.coordinateUnits = UNIT.METER; init_parameters.depthMinimumDistance = 0.15f ; // Set the minimum depth perception distance to 15cm
- Lowering the minimum range to very small values can dramatically increase memory requirements and reduce FPS. Increase this value to improve performance.
- For applications requiring long-range depth perception, we recommend setting
depth_minimum_distanceto 1m or more for improved performance.
Maximum Range #
The maximum range can be increased using
InitParameters init_parameters; init_parameters.depth_mode = DEPTH_MODE::ULTRA ; // Set the depth mode to ULTRA init_parameters.coordinate_units = UNIT::METER; init_parameters.depth_maximum_distance = 40; // Set the maximum depth perception distance to 40m
init_params = sl.InitParameters() init_parameters.depth_mode = sl.DEPTH_MODE.ULTRA # Set the depth mode to ULTRA init_parameters.coordinate_units = UNIT.METER init_parameters.depth_maximum_distance = 40 # Set the maximum depth perception distance to 40m
InitParameters init_parameters = new InitParameters(); init_parameters.depthMode = DEPTH_MODE.ULTRA ; // Set the depth mode to ULTRA init_parameters.coordinateUnits = UNIT.METER; init_parameters.depthMaximumDistance = 40; // Set the maximum depth perception distance to 40m
- We recommend using the
ULTRAdepth mode to improve depth accuracy at long distances.
- The maximum depth range can be reduced to clamp values above a certain distance using
sl::InitParameters::depth_maximum_distance. This is useful to reduce depth jitter at long distances.
- Increasing the maximum range has no impact on memory or FPS.
Depth Stabilization #
Depth stabilization is a feature that temporally fuses and filters the depth map over several frames. This allows the reduction of jitter and improves depth accuracy on static objects. Depth stabilization works even when the camera is moving by using the positional tracking capability of the ZED SDK. It can also detect moving objects to avoid fusing the depth of dynamic areas.
Depth stabilization is enabled by default. Since it enables positional tracking in the background, you can disable depth stabilization using
init_parameters.depth_stabilization = false to improve computational performance.
Tips: For fixed cameras, we recommend enabling
PositionalTrackingParameters::set_as_staticwhile using depth stabilization. This allows the depth stabilizer module to know the camera is static so it can disable visual tracking and reduce the computational load.
Depth Accuracy #
Stereo vision uses triangulation to estimate depth from a disparity image, with the following formula describing how depth resolution changes over the range of a stereo camera:
Dr is depth resolution,
Z the distance and
alpha a constant.
Depth accuracy decreases quadratically over the z-distance, with a stereo depth accuracy of 1% of the distance in the near range to 9% in the far range. Depth accuracy can also be affected by outliers’ measurements on homogenous and textureless surfaces such as white walls, green screens and specular areas. These surfaces usually generate temporal instability in the depth measurements.
Best Practices #
All cameras from Stereolabs can be used both indoors and outdoors at long ranges (up to 20m). The configuration that offers the best depth accuracy is to place the camera at a working distance of 30cm to 1 meter from the scene. To capture fast movements, use the camera’s high-speed modes (HD720 @ 60 FPS or VGA @ 100 FPS).
Avoid using the camera in very low-light environments as the camera uses color images for depth perception. However, you can improve low-light performance by reducing the camera frame rate to 15FPS.
You can also extend the range of the USB cable up to 15m by using a USB 3.0 extension cable and up to 100m using an optical fiber USB 3.0 extender.