Basic Concepts

This section serves as an introduction to the ZED Plugin for Unity. It details some of the plugin’s basic concepts and features which you will find useful in most projects.


When you create a new project and import ZEDCamera.package, you will find a Mono and Stereo camera rig in the Prefabs folder: ZED_Rig_Mono and ZED_Rig_Stereo. These prefabs are custom AR cameras that replace the regular Unity Camera in a scene. They mix the virtual 3D elements rendered by the Unity camera with the real-world video captured by the ZED.

Attached to the ZED Rig Mono is Camera_Left which contains a Frame holding the video source of the left camera. The ZED Rig Stereo prefab contains both the left and right video sources for passthrough AR. When adding the ZED Rig to your hierarchy, it will appear at (0,0,0) with -Z representing the forward direction facing the camera.

Note: When adding ZED Rig to your project, delete the Main Camera from the Hierarchy to avoid interfering with the embedded camera of the prefab. If your use case requires multiple camera outputs in your scene, you may need multiple display windows visible in the editor.

Depth-aware AR

The ZED and ZED Mini cameras capture depth and normal maps of the surrounding environment in real-time. To create a believable and interactive AR/MR experience, the depth and normal buffers captured by the cameras are integrated into the Unity rendering pipeline. This allows to add several key AR features:

  • Object placement: Virtual objects can be placed anywhere in the real world without having to scan the environment first. See Object Placement to learn more.
  • Interactive collisions: Virtual elements can collide with real people and objects moving in the camera field of view.
  • Realistic shadows and lighting: Unity’s lights can cast shadows and project light onto the real world. See Lighting and Shadows to learn how to enable interactive AR lighting.
  • Depth occlusions: Virtual objects are naturally occluded by the surrounding environment. Occlusions are automatically enabled by the plugin.

The features above are accessible through the ZED Manager or support functions located in the Scripts folder of the package. Explore the different samples to learn how to use these support functions.

ZED Manager

The ZED Manager allows you to configure video, depth and tracking parameters. These parameters are set by default and you often don’t need to modify them when adding a rig prefab to your project.

After adding a ZED_Rig_mono or ZED_Rig_Stereo to the Hierarchy view, click on it to access the parameters of the ZED Manager script in the Inspector window.


  • Resolution: Video mode of the ZED camera. Higher resolutions result in lower FPS. See Video Modes.
  • Depth mode: Depth quality mode. PERFORMANCE mode is recommended. See Depth Modes.

Motion Tracking

  • Enable Tracking: If positional tracking is enabled, the virtual rig automatically moves to match the real-world movement of the device using its own input (no need for an external tracker). This allows you to place virtual objects in the scene and make sure they keep their real-world position while you move. Disable this option when using external sensors for tracking. See Positional Tracking.
  • Enable Spatial Memory: Enable spatial memory to correct drift during tracking. There can be small pose jumps when a correction is required. See Spatial Memory.
  • Path Spatial Memory: Path to the file that will be loaded and/or saved by the Spatial Memory. Loading an existing file will allow absolute localization in an environment. Leaving blank will not save an area file at the end of a tracking session.


  • Video Overlay: Overlay real-world video with virtual elements rendered by the ZED Rig camera (for AR/MR).
  • Camera Brightness: Control image brightness. Use this to darken the real-world video without affecting virtual objects.
  • AR Post-processing: Enable post-processing of the virtual layer to match the real camera characteristics.
  • Rendering Path: Select Forward or Deferred rendering path. Use Forward for better performance in MR.
  • Camera Control: Open control panel to adjust Camera Settings.

Advanced Settings

  • ZED Plugin Layers: The plugin uses four layers to make sure that its in-game cameras can only see images it’s supposed to. For example, the left Unity camera should not see the plane that displays the feed from the ZED’s right camera. You can change the layers the plugin uses here in case you are already using the default layers for something else.
  • Show Final AR Rig: In pass-through AR mode, the plugin uses a second, hidden AR rig to make final adjustments to the image before it’s sent to the headset. It’s hidden by default because it can be quite confusing to see, but this setting lets advanced users observe and modify this rig.
  • Fade In At Start: Disable to remove the fade-in effect you see when the ZED is first connected.
  • Don’t Destroy On Load: Enable to set the ZED rig’s DontDestroyOnLoad value, which will prevent its destruction when you change scenes.

Using SVO files

During application development, it can be easier to work with a recorded video instead of a live video feed. In Unity, you can record and read SVO files by selecting ZED_Rig_Mono or ZED_Rig_Stereo in the Hierarchy panel, then Add Component > SVO Manager in the Inspector panel.

  • Record: If enabled, will start recording an SVO file when the ZED is first connected until the ZZED becomes disconnected. Can’t be enabled when Read is enabled.
  • Read:: If enabled, will play the SVO file and use it as input instead of any physical ZED’s connected to your PC.
  • Loop: Causes the SVO file to start over once it reaches the end. Tracking is not supported when this is enabled.
  • Compression Mode: If recording, this defines what kind of compression is used on the file. Note that uncompressed SVO files are extremely large (multiple gigabytes per minute).
  • Realtime Playback: If enabled when Reading an SVO, the time between each frame is defined by the timestamps recorded onto each frame. Otherwise, each frame is played at the normal rate defined by the resolution (ex. 60fps for 720). This mainly affects dropped frames: When enabled, dropped frames will cause a slight pause. When disabled, dropped frames will cause a “jump.”