Setting up ZED in Isaac Sim
This tutorial guides you through setting up the ZED X camera in NVIDIA® Isaac Sim and connecting to the ZED SDK. If you’re not familiar with Isaac Sim, make sure to go through the Isaac Sim Official documentation.
Setting up your virtual camera #
Add the model to the stage #
The ZED X model is available in the Isaac Assets tab on the panel or in the extension repository under the
usd folder. Simply drag and drop it onto the scene.
You will notice the prim has a base_link XForm, this is the reference frame of the mounting point on the bottom of the camera.
Activate the ZED Camera extension #
The virtual camera takes advantage of the ZED SDK streaming input to stream its data into the ZED SDK.
The first step is to enable the ZED Camera extension to get access to the ZED Camera Streamer Omnigraph node.
In the top menu bar, navigate to Window -> Extension to show the extension manager.
Alternatively, you can clone the extension repository and add it to the list of extensions loaded in the Extension manager. Click on the settings button and create a new entry with the path to the root folder of the cloned repository.
Build the action graph #
The streaming is activated using the Omnigraph node ZED Camera streamer. To stream data at each simulation tick, drag and drop this node into the graph (you can type zed in the search panel to quickly find it), add its ZED Camera prim target to the camera in the scene, and connect its ExecIn input to the Tick output of the On Playback Tick node.
You can also configure the streamer properties in the Property panel on the right:
- Serial number: this field allows you to identify the virtual camera you are using. It has no functional effect and can be left to its default value. This field can be useful when using multiple cameras to differentiate them.
- Streaming port: Defines which port is used for streaming. It must be an even number.
- Use system time: when set to true, the streamer assigns the latest system time to the images that are sent to the SDK. If set to false, the streamer only sets the first timestamp to system time and then increments the next timestamps by simulation time.
- ZED Camera prim: this parameter should be set to the ZED X camera prim in the stage as shown in the picture above.
And that’s it, you can now start the simulation to stream your stage into the ZED SDK!
Test it in ZED DepthViewer #
Once streaming is started, you can test it by connecting the ZED DepthViewer to the virtual camera. Simply launch the tool, click on the streaming input button, set the IP Address (the streaming computer) and the port (set in Streaming port), and you should see the depth data coming from the virtual environment.