Depth Modes

Stereolabs AI Depth leverages advanced neural networks to generate high-quality depth maps from stereo images, delivering reliable results even in challenging scenarios. Compared to traditional approaches, AI Depth provides superior accuracy in low-texture and low-light environments. This makes it especially well-suited for applications such as robotics, augmented reality (AR), and 3D mapping, where dependable depth perception is critical.

The ZED SDK provides multiple AI-powered depth modes, allowing you to tailor depth sensing to your application’s requirements. Each mode offers a different balance of accuracy, range, and computational speed, so you can optimize for precision, performance, or a mix of both depending on your use case.

NEURAL #

The NEURAL depth mode uses AI-powered disparity estimation to deliver a strong balance of depth accuracy and processing speed. It is ideal for applications that require reliable depth perception without sacrificing real-time performance.

Neural Depth Computational Performances on embedded devices #

• Orin AGX
• Orin NX 16
• Orin NX 8
• Nano 8 Gb
• Nano 4 Gb

Cameras	FPS	CPU (%)	GPU (%)
1	30	5	26
	2	30	6	50
		4	30	26	53

Cameras	FPS	CPU (%)	GPU (%)
1	30	9	59
	2	23	20	94
		4	10	39	96

Cameras	FPS	CPU (%)	GPU (%)
1	12	22	90
	2	5	25	93
		4	OOM	OOM	OOM

Cameras	FPS	CPU (%)	GPU (%)
1	30	14	88
	2	18	20	98
		4	9	38	98

Cameras	FPS	CPU (%)	GPU (%)
1	30	12	70
	2	17	18	96
		4	6	35	98

📌 Note: performance obtained with ZED SDK v5.0.1 RC, ZED X Driver v1.3.0, and ZED X camera using the multicamera code example available on GitHub.

Neural Depth Accuracy (ZED X) #

Distance Range (m)	Mean Error	Standard Deviation*
[0.3 - 4]	< 1 %	Low
[4 - 6]	< 2.5%	Low
[6 - 9 ]	< 4%	Medium
[10 - 12 ]	< 6%	High

(*) A lower standard deviation indicates more stable and accurate depth estimation, resulting in smoother and more reliable 3D point clouds. Higher deviation can lead to noise and distortion, producing wavy or unstable point clouds.

Enabling the NEURAL depth mode in the API #

• C++
• Python
• C#

// Set depth mode in NEURAL
InitParameters init_parameters;
init_parameters.depth_mode = DEPTH_MODE::NEURAL;

# Set depth mode in NEURAL
init_parameters = sl.InitParameters()
init_parameters.depth_mode = sl.DEPTH_MODE.NEURAL

// Set depth mode in NEURAL
InitParameters init_parameters = new InitParameters();
init_parameters.depthMode = DEPTH_MODE.NEURAL;

NEURAL LIGHT #

The NEURAL_LIGHT depth mode provides AI-powered disparity estimation optimized for speed and efficiency. It enables real-time depth sensing with lower computational load, making it ideal for multi-camera setups and applications where fast processing is prioritized over maximum depth accuracy.

Neural Light Depth Computational Performances on embedded devices #

• Orin AGX
• Orin NX 16
• Orin NX 8
• Nano 8 Gb
• Nano 4 Gb

Cameras	FPS	CPU (%)	GPU (%)
1	30	5	11
	2	30	6	23
		4	30	22	46

Cameras	FPS	CPU (%)	GPU (%)
1	30	2	23
	2	30	5	47
		4	30	14	81

Cameras	FPS	CPU (%)	GPU (%)
1	30	24	80
	2	13	27	80
		4	OOM	OOM	OOM

Cameras	FPS	CPU (%)	GPU (%)
1	30	14	36
	2	30	25	64
		4	21	45	84

Cameras	FPS	CPU (%)	GPU (%)
1	30	15	28
	2	30	25	55
		4	19	40	90

📌 Note: performance obtained with ZED SDK v5.0.1 RC, ZED X Driver v1.3.0, and ZED X camera using the multicamera code example available on GitHub.

Neural Depth Accuracy (ZED X) #

Distance Range (m)	Mean Error	Standard Deviation*
[0.3 - 3]	< 1 %	Low
[3 - 5]	< 3%	Medium
[5 - 12 ]	< 8%	High

(*) A lower standard deviation indicates more stable and accurate depth estimation, resulting in smoother and more reliable 3D point clouds. Higher deviation can lead to noise and distortion, producing wavy or unstable point clouds.

Enabling the NEURAL LIGHT depth mode in the API #

• C++
• Python
• C#

// Set depth mode in NEURAL_LIGHT
InitParameters init_parameters;
init_parameters.depth_mode = DEPTH_MODE::NEURAL_LIGHT;

# Set depth mode in NEURAL_LIGHT
init_parameters = sl.InitParameters()
init_parameters.depth_mode = sl.DEPTH_MODE.NEURAL_LIGHT

// Set depth mode in NEURAL_LIGHT
InitParameters init_parameters = new InitParameters();
init_parameters.depthMode = DEPTH_MODE.NEURAL_LIGHT;

NEURAL PLUS #

The NEURAL_PLUS depth mode provides the highest depth accuracy and detail among all AI-powered modes. It is designed for applications that demand maximum precision and robustness, such as advanced robotics, inspection, and 3D reconstruction. While it requires more computational resources and delivers lower frame rates compared to other modes, NEURAL_PLUS excels in challenging environments and when capturing fine object details is critical.

Neural Plus Depth Computational Performances on embedded devices #

• Orin AGX
• Orin NX 16
• Orin NX 8
• Nano 8 Gb
• Nano 4 Gb

Cameras	FPS	CPU (%)	GPU (%)
1	29	7	90
	2	17	11	90
		4	8	21	97

Cameras	FPS	CPU (%)	GPU (%)
1	12	2	92
	2	5	4	98
		4	2	14	98

Cameras	FPS	CPU (%)	GPU (%)
1	3	20	97
	2	1.3	25	98
		4	OOM	OOM	OOM

Cameras	FPS	CPU (%)	GPU (%)
1	8	12	95
	2	4	18	97
		4	2	35	98

Cameras	FPS	CPU (%)	GPU (%)
1	8	10	94
	2	3	19	98
		4	1	34	98

📌 Note: performance obtained with ZED SDK v5.0.1 RC, ZED X Driver v1.3.0, and ZED X camera using the multicamera code example available on GitHub.

Neural Plus Accuracy (ZED X) #

Distance Range (m)	Mean Error	Standard Deviation*
[0.3 - 9]	< 1 %	Low
[9 - 12]	< 2%	Medium

(*) A lower standard deviation indicates more stable and accurate depth estimation, resulting in smoother and more reliable 3D point clouds. Higher deviation can lead to noise and distortion, producing wavy or unstable point clouds.

Enabling the NEURAL PLUS depth mode in the API #

• C++
• Python
• C#

// Set depth mode in NEURAL_PLUS
InitParameters init_parameters;
init_parameters.depth_mode = DEPTH_MODE::NEURAL_PLUS;

# Set depth mode in NEURAL_PLUS
init_parameters = sl.InitParameters()
init_parameters.depth_mode = sl.DEPTH_MODE.NEURAL_PLUS

// Set depth mode in NEURAL_PLUS
InitParameters init_parameters = new InitParameters();
init_parameters.depthMode = DEPTH_MODE.NEURAL_PLUS;

Depth Modes Comparison #

Depth Mode	Ideal Range	Benefits	Limitations
NEURAL_LIGHT	[0.3-5]	Fastest depth mode available Best for multi camera setup Suited for mid-range obstacle avoidance	Smallest ideal depth range May miss small objects or objects details Slightly less robust to environmental light changes than NEURAL
	NEURAL	[0.3-9]	Balanced depth and performance Better object detail than NEURAL_LIGHT Suitable for most multi-camera applications Same robustness to environmental changes as NEURAL_PLUS	Slower than NEURAL_LIGHT Less detail than NEURAL_PLUS
		NEURAL_PLUS	[0.3-12]	Highest object details available Highest ideal depth range and stability Best for detecting near, far, and small objects Most robust to environmental changes (rain,sun) and light reflections	Slowest depth mode May not be suited for multi camera setup

Note:

• The depth range is highly dependent on the camera baseline and optics. A bigger baseline produces increased depth range. Here, tests were conducted with a ZED X GS (lens of 2 mm) whose stereo baseline is of 120 mm.
• Jetson Power Profile: Tests were conducted using MAXN without Super mode.