Getting started with Isaac Sim

NVIDIA® Isaac sim is part of the NVIDIA® Omniverse platform, it is a high-fidelity photo-realistic simulator tailored to robotics simulation. This tutorial covers the essential steps needed to get started with Isaac Sim and the ZED. For a more in-depth tutorial, you can follow the official guides from Nvidia.

Install NVIDIA® Isaac Sim #

The recommended way to install Isaac Sim is to use the Omniverse launcher. If you are installing it on Linux it is recommended to use the recommended NVIDIA® GPU Driver version shown in the Isaac Sim requirements.

The Omniverse launcher allows the installation of Omniverse applications. We recommend installing the following applications for a ready-to-start configuration:

For more information follow the Official NVIDIA® guide.

Workstation Setup #

Launch Isaac Sim #

Isaac Sim can be executed locally on your computer or on a remote station (Headless mode).

When it is launched on a remote station, it will start streaming its data so any computer on the same network can connect to it and see its Graphical User Interface. This interface is accessible via a web browser or the Omniverse Streaming client.

  • Start Omniverse Launcher and select the LIBRARY tab.
  • In the left column, select “Isaac Sim”.
  • Hit the LAUNCH button to start the application.
  • You will be prompted to select a launch mode, the headless modes are either “WebRTC” (web browser) or “Native” (Omniverse streaming client).

Note: The Omniverse Streaming Client can be installed via the Omniverse Launcher.

Create your environment #

The environment in Isaac Sim consists of a Stage (panel to the right), that holds all of the assets in the USD Scene. To have a working simulation with physics, you need to:

  • Create a physics scene that allows you to create physics and gravity.
  • Add a ground plane that prevents objects from falling infinitely.

A quick way to explore these things is to use a predefined environment by going to the top menu bar Create -> Isaac -> Environments and choosing the Flat grid as it is the simplest one.

You will notice that the stage has been populated by items with different prims (Prim is short for primitive, the fundamental container object in Omniverse). The window that shows the flat grid is called the Viewport.

Add objects to the scene #

You can use the Content and Isaac Assets panels located at the bottom of the window to add prims to the stage.

The Content tab lets you look for files located on your machine while the Isaac Assets tab shows all of the available assets in Isaac Sim. To add an item simply drag it into your scene or the stage panel.

Create an Action graph #

Action graphs are part of the Visual Scripting tools provided by Isaac Sim, allowing for programming using nodes instead of code. They are event-based and enable the user to execute actions based on a sequence of events and conditions.
To create an Action graph, right-click anywhere in the viewport and select Create -> Visual Scripting -> Action graph

A new action graph appeared in the Stage view and the relative panel Action Graph is now open at the bottom of the screen.

Start typing “OnPlaybackTick” in the search bar of the left side of the panel to find the node. Then drag and drop it into the graph. Same thing for the “Print Text” node.

Connect the Tick of the OnPlaybackTick to the In of the Print text. Then click on the Print text node, and fill out its properties in the Property panel on the right.

You can experiment with different settings to print on the screen (viewport) or on the Console tab. To execute the graph, simply press play (to trigger a playback tick) and you should see the printed text.

For a detailed explanation of Omnigraph, Omniverse’s visual programming framework, refer to the official documentation that shows you how to control a robot using action graphs.

Now that we have a solid grasp of Isaac Sim’s core concepts and interface, we are ready to configure a ZED camera within the Isaac SIM environment.