Getting started with Isaac Sim
NVIDIA® Isaac sim is part of the NVIDIA® Omniverse platform, it is a high-fidelity photo-realistic simulator tailored to robotics simulation. This tutorial covers the essential steps needed to get started with Isaac Sim and the ZED. For a more in-depth tutorial, you can follow the official guides from Nvidia.
Install NVIDIA® Isaac Sim #
The recommended method to install Isaac Sim is to use the Omniverse launcher. If you are installing it on Linux, we advise using the NVIDIA® GPU Driver version specified in the Isaac Sim requirements.
The Omniverse launcher allows the installation of Omniverse applications. At least installing Nucleus, Cache, and Isaac Sim is recommended for the best experience.
- Nucleus allows you to save and share assets in Omniverse applications.
- Cache is a very useful application that allows you to speed up asset imports by caching data.
- Isaac Sim is the simulator will be using, it is recommended to read the Isaac Sim Main Concepts documentation to get familiar with it.
For more information follow the Official NVIDIA® guide.
Launch Isaac Sim #
Isaac Sim can be launched locally on your computer or on a remote station (Headless). When it is launched on a remote station, it will start streaming its data so any computer on the same network can connect to it and see its Graphical User Interface. This interface is accessible via a web browser or the Omniverse Streaming client.
Isaac Sim can be launched by going into the Library tab in Omniverse launcher -> Isaac Sim -> Launch. You will be prompted to select a launch mode, the headless modes are either “WebRTC” (web browser) or “Native” (Omniverse streaming client).
Note: The Streaming Client can be installed via the Omniverse Launcher.
Create your environment #
The environment in Isaac Sim consists of a Stage (panel to the right), that holds all of the assets in the USD Scene. To have a working simulation with physics, you need to:
- Create a physics scene that allows you to create physics and gravity.
- Add a ground plane that prevents objects from falling infinitely.
A quick way to explore these things is to use a predefined environment by going to the top menu bar Create -> Isaac -> Environments and choosing the Flat grid as it is the simplest one.
You will notice that the stage has been populated by items with different prims (Prim is short for primitive, the fundamental container object in Omniverse). The window that shows the flat grid is called the Viewport.
Add objects to the scene #
You can use the Content and Isaac Assets panels located at the bottom of the window to add prims to the stage. The Content tab lets you look for files located on your machine while the Isaac Assets tab shows all of the available assets in Isaac Sim.
To add an item simply drag it into your scene or the stage panel.
Create an Action graph #
Action graphs are part of the Visual Scripting tools provided by Isaac Sim, allowing for programming using nodes instead of code. They are event-based and enable the user to execute actions based on a sequence of events and conditions.
To create an Action graph, right-click anywhere in the viewport and select Create -> Visual Scripting -> Action graph
You will notice that the action graph appeared in the stage and that its panel Action Graph is now open at the bottom of the screen. Start typing “OnPlaybackTick” on the left side of the panel to find the node. Then drag and drop it into the graph. Same thing for the “Print Text” node.
Finally, connect the Tick of the OnPlaybackTick to the In of the Print text. Then click on the Print text node, and fill out its properties in the Property panel on the right. You can experiment with different settings to print on the screen (viewport) or on the Console tab. To execute the graph, simply press play (to trigger a playback tick) and you should see the printed text.
For a detailed explanation of Omnigraph, Omniverse’s visual programming framework, refer to the official documentation that shows you how to control a robot using action graphs.