Setting up Isaac™ ROS to work with ZED Cameras in ROS 2

This guide will help you set up the NVIDIA® Isaac™ ROS packages to work with the ZED ROS 2 Wrapper.

📌 Note: A solid connection to the internet is required to download the NVIDIA® Isaac™ ROS packages and the StereoLabs software If you are using a Jetson device, we recommend using a wired connection for better stability and speed.

NVIDIA® Jetson™ Setup #

📌 Note: you can ignore this section if you are not using an NVIDIA® Jetson™ device.

Before installing Isaac™ ROS on an NVIDIA® Jetson™ device, ensure that it is set up correctly and has the necessary software installed.

Install the prerequisites #

Open a terminal [Ctrl+Alt+T] and enter the following commands to install the prerequisites:

sudo apt update
sudo apt install python3 python3-pip # Install python3
sudo apt install nvidia-jetpack # Install the Jetson developer tools
sudo pip3 install jetson-stats # Optional but recommended to monitor the status of the Jetson™ module

Install the VPI library #

Configure the VPI library for running compute on Jetson™ device’s PVA accelerator.

  1. Generate CDI Spec for GPU/PVA:

    sudo nvidia-ctk cdi generate --mode=csv --output=/etc/cdi/nvidia.yaml

  2. Install pva-allow-2 package:

    # Add Jetson public APT repository
sudo apt-get update
sudo apt-get install software-properties-common
sudo apt-key adv --fetch-key https://repo.download.nvidia.com/jetson/jetson-ota-public.asc
sudo add-apt-repository 'deb https://repo.download.nvidia.com/jetson/common r36.4 main'
sudo apt-get update
sudo apt-get install -y pva-allow-2

Install an additional SSD memory if you are using an NVIDIA® Jetson™ development kit #

Using Isaac™ ROS with Docker requires a fast and large storage solution, such as an SSD, to ensure optimal performance.

The Docker images can be quite large, and running them on slower storage devices may lead to performance issues.

If you are using an NVIDIA® Jetson™ development kit device, we recommend you follow the official NVIDIA® documentation for best practices on setting up a NVMe SSD card in the Jetson developer kit’s carrier board.

If you are using instead a StereoLabs ZED Box, ZED Box Mini, or a Mini Carrier board, the preinstalled NVMe SSD is already configured for optimal performance and no further configuration is required.

Configure NVIDIA® Isaac™ ROS using Docker #

📌 Note: You can skip this section if you are not using Docker to run your Isaac™ ROS applications and prefer to use the official Apt Repository.

This setup is recommended when using an NVIDIA® Jetson™ device.

NVIDIA® provides ready-to-use Docker images for using Isaac™ ROS. The Docker configuration allows you to easily set up your development environment with all the necessary dependencies and tools pre-installed on Jetson devices and x86_64 systems.

Please refer to the official NVIDIA® Isaac™ ROS documentation for advanced configurations and usage.

  1. Install the pre-requisite:

    sudo apt-get install python3-pip # Install python3

  2. Restart Docker:

    sudo systemctl daemon-reload && sudo systemctl restart docker

  3. Install Git LFS to pull down all large files:

    sudo apt-get install git-lfs
git lfs install  --skip-repo

  4. Create a workspace and add it to the environment:

    mkdir -p ~/ros2/isaac_ros_ws/src
echo "export ISAAC_ROS_WS=${HOME}/ros2/isaac_ros_ws/" >> ~/.bashrc
source ~/.bashrc

  5. Enter the workspace and clone packages:

    cd ${ISAAC_ROS_WS}/src
git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git --branch release-3.2 --recurse-submodules

  6. Pull the official Isaac Common Docker image and start the first instance of development environment:

    cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
./scripts/run_dev.sh

For any problems with the installation please refer to the NVIDIA® Isaac™ ROS Troubleshooting documentation.

Configure NVIDIA® Isaac™ ROS using Docker with ZED cameras #

To use Isaac™ ROS with ZED cameras, you must create a Docker layer that installs the ZED SDK in the Isaac™ ROS Docker image. This layer will allow you to use ZED cameras in your Isaac™ ROS applications.

  1. Create a new Docker.zed file in the isaac_ros_common/docker directory:

    cd ${ISAAC_ROS_WS}/src/isaac_ros_common/docker
touch Docker.zed

  2. Open the Docker.zed file in a text editor and add the following content:

    ARG BASE_IMAGE
FROM ${BASE_IMAGE} AS catscanners

# disable terminal interaction for apt
ENV DEBIAN_FRONTEND=noninteractive
ENV SHELL=/bin/bash
SHELL ["/bin/bash", "-c"]

ARG ARCH=aarch64
#ARG ARCH=x86_64

# Copy the SDK install script
COPY scripts/install-zed-${ARCH}.sh /opt/zed/install-zed.sh

RUN chmod +x /opt/zed/install-zed.sh && \
   /opt/zed/install-zed.sh

    📌 Note: Uncomment the ARG ARCH=x86_64 line if you are using an x86_64 system instead of a Jetson™ device.

  3. Modify the install-zed-<arch>.sh file to install the latest ZED SDK.

    A. Open the isaac_ros_common/docker/scripts/install-zed-<arch>.sh file in a text editor

    B. Search for the line that starts with wget -q and replace it with the following lines to download the latest ZED SDK and then check the validity of the download:

    Jetson:

    URL=https://download.stereolabs.com/zedsdk/5.0/l4t36.4/jetsons
wget -q --no-check-certificate -O ZED_SDK_Linux.run ${URL}
SIZE=$(du -sb './ZED_SDK_Linux.run' | awk '{ print $1 }')

if ((SIZE<1)) ; then
   echo "ERROR: ZED_SDK_Linux.run is empty!!!";
   echo " * Verify the ZED SDK download link validity: " ${URL} ;
   exit 1
fi
    Full "scripts/install-zed-aarch64.sh" content
    # Download dependencies for zed SDK installation RUN file
sudo apt-get update -y || true
sudo apt-get install --no-install-recommends lsb-release wget less zstd udev sudo apt-transport-https -y

# Download zed SDK installation RUN file to /tmp directory
cd /tmp

URL=https://download.stereolabs.com/zedsdk/5.0/l4t36.4/jetsons
wget -q --no-check-certificate -O ZED_SDK_Linux.run ${URL}
SIZE=$(du -sb './ZED_SDK_Linux.run' | awk '{ print $1 }')

if (($SIZE==0)) ; then
   echo "ERROR: ZED_SDK_Linux.run is empty!!!";
   echo " * Verify the ZED SDK download link validity: " ${URL} ;
   exit 1
fi

sudo chmod 777 ./ZED_SDK_Linux.run
sudo -u admin ./ZED_SDK_Linux.run silent skip_od_module skip_python skip_drivers
# Symlink required to use the streaming features on Jetson inside a container, based on
# https://github.com/stereolabs/zed-docker/blob/fd514606174d8bb09f21a229f1099205b284ecb6/4.X/l4t/devel/Dockerfile#L27C5-L27C95
sudo ln -sf /usr/lib/aarch64-linux-gnu/tegra/libv4l2.so.0 /usr/lib/aarch64-linux-gnu/libv4l2.so

# Install zed-ros2-wrapper dependencies
sudo apt update && sudo apt-get install --no-install-recommends ros-humble-zed-msgs -y

# Cleanup
sudo rm -rf /usr/local/zed/resources/*
rm -rf ZED_SDK_Linux.run
sudo rm -rf /var/lib/apt/lists/*

    📌 Note: If you are using a different version of the JetPack SDK, you may need to adjust the URL accordingly. For example, if you are using JetPack 6.0, you must change l4t36.4 to l4t36.3.

    x86_64:

    URL=https://download.stereolabs.com/zedsdk/5.0/cu12/ubuntu24
wget -q --no-check-certificate -O ZED_SDK_Linux.run ${URL}
SIZE=$(du -sb './ZED_SDK_Linux.run' | awk '{ print $1 }')

if ((SIZE<1)) ; then
   echo "ERROR: ZED_SDK_Linux.run is empty!!!";
   echo " * Verify the ZED SDK download link validity: " ${URL} ;
   exit 1
fi
    Full "scripts/install-zed-x86_64.sh" content
    # Download dependencies for zed SDK installation RUN file
sudo apt-get update -y || true
sudo apt-get install --no-install-recommends lsb-release wget less udev sudo zstd build-essential cmake libpng-dev libgomp1 -y

# Download zed SDK installation RUN file to /tmp directory
cd /tmp
URL=https://download.stereolabs.com/zedsdk/5.0/cu12/ubuntu24
wget -q --no-check-certificate -O ZED_SDK_Linux.run ${URL}
SIZE=$(du -sb './ZED_SDK_Linux.run' | awk '{ print $1 }')

if ((SIZE<1)) ; then
   echo "ERROR: ZED_SDK_Linux.run is empty!!!";
   echo " * Verify the ZED SDK download link validity: " ${URL} ;
   exit 1
fi

sudo chmod 777 ./ZED_SDK_Linux.run
sudo -u admin ./ZED_SDK_Linux.run silent skip_od_module skip_python skip_drivers


# Install zed-ros2-wrapper dependencies
sudo apt update && sudo apt-get install --no-install-recommends ros-humble-zed-msgs -y

# Cleanup
sudo rm -rf /usr/local/zed/resources/*
rm -rf ZED_SDK_Linux.run
sudo rm -rf /var/lib/apt/lists/*

  4. Build the Docker image with the ZED SDK installed:

    cd ${ISAAC_ROS_WS}/src/isaac_ros_common
./scripts/run_dev.sh -i ros2_humble.zed

    Wait for the build to complete. This may take some time ☕

  5. When the build is complete, the Docker container will start automatically. You can check that the ZED SDK is installed by running the following commands inside the Docker container:

    Check the ZED SDK version:

    /opt/zed/tools/ZED_Explorer --version

    Check camera opening and Depth extraction capabilities:

    /opt/zed/tools/ZED_Depth_Viewer

Configure NVIDIA® Isaac™ ROS from the official Apt Repository #

📌 Note: you can skip this section if you are using Docker to run your Isaac™ ROS applications.

On Desktop machines, you can easily install the NVIDIA® Isaac™ ROS packages using the official Apt Repository. This repository provides pre-built packages for all supported platforms, making it easy to get started with Isaac™ ROS without using Docker.

📌 Note: We do not recommend this procedure when working with Jetson devices because it’s not straightforward to satisfy all dependencies.

NVIDIA® Isaac™ Apt Repository built on NVIDIA’s CDN (Content Distribution Network) hosts the following categories of packages for all users.

  • All Isaac™ ROS packages pre-built for the currently supported platforms.
  • Auxiliary packages to improve developer experience or to support a legacy OS.

Set up the Isaac™ Apt Repository** #

Open a terminal and follow these steps to set up the NVIDIA® Isaac™ ROS Apt Repository on your system. This guide assumes you are using Ubuntu 22.04.

  1. Set Locale - Ensure your system locale is set to UTF-8:

    locale  # check for UTF-8

sudo apt update && sudo apt install locales
sudo locale-gen en_US en_US.UTF-8
sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
export LANG=en_US.UTF-8

locale  # verify settings

  2. Install Dependencies - Install the required dependencies for the Isaac™ Apt Repository:

    sudo apt update && sudo apt install gnupg wget
sudo apt install software-properties-common
sudo add-apt-repository universe

  3. Setup Source - Register the GPG key with apt. Two options are listed below, one for .com (US CDN) and one for .cn (China CDN) - choose whichever you prefer.

    wget -qO - https://isaac.download.nvidia.com/isaac-ros/repos.key | sudo apt-key add -
grep -qxF "deb https://isaac.download.nvidia.com/isaac-ros/release-3 $(lsb_release -cs) release-3.0" /etc/apt/sources.list || \
echo "deb https://isaac.download.nvidia.com/isaac-ros/release-3 $(lsb_release -cs) release-3.0" | sudo tee -a /etc/apt/sources.list
sudo apt-get update

    wget -qO - https://isaac.download.nvidia.cn/isaac-ros/repos.key | sudo apt-key add -
grep -qxF "deb https://isaac.download.nvidia.cn/isaac-ros/release-3 $(lsb_release -cs) release-3.0" /etc/apt/sources.list || \
echo "deb https://isaac.download.nvidia.cn/isaac-ros/release-3 $(lsb_release -cs) release-3.0" | sudo tee -a /etc/apt/sources.list
sudo apt-get update

  4. Add the NVIDIA Keyring - Register the NVIDIA GPG key with apt:

    sudo apt update && sudo apt install curl -y
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(. /etc/os-release && echo $UBUNTU_CODENAME) main" | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null

You are now ready to install the NVIDIA® Isaac™ ROS packages.

Install the required NVIDIA® Isaac™ ROS packages #

Open a terminal and run the following command to install the required NVIDIA® Isaac™ ROS packages:

sudo apt update
sudo apt install ros-humble-isaac-ros-managed-nitros ros-humble-isaac-ros-nitros-image-type

Install the ZED SDK and ZED X Driver #

If you haven’t already, install the ZED SDK and the ZED X Driver on your system. The ZED X Driver is required to use ZED X, ZED X Mini, and ZED X One cameras with Isaac™ ROS.

Install the ZED ROS 2 Wrapper packages #

You can now install the ZED ROS 2 Wrapper packages with NITROS support to be used in your Isaac™ ROS applications by following the instructions in the ZED ROS 2 Wrapper Installation Guide. The ZED ROS 2 Wrapper will automatically detect the available NVIDIA® Isaac™ ROS packages and enable NITROS support during the build process.

If you are new to the ZED ROS 2 Wrapper, we recommend starting with the Getting Started with ROS 2 and ZED guide. This guide will help you set up your environment and run your first ZED camera application using ROS 2.

You can also explore the ZED ROS 2 Examples repository for tutorials and advanced examples.

📌 Note: If the ZED ROS 2 Wrapper was already installed in your workspace, just clean and rebuild your workspace to ensure that the ZED ROS 2 Wrapper is built with the NVIDIA® Isaac™ ROS support.

cd ~/your_workspace # e.g. cd ~/devel/ros_ws
rm -rf build/ install/ log/
colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release
source install/setup.bash

You can verify that the ZED ROS 2 Wrapper is built with NVIDIA® Isaac™ ROS support by checking the output of the colcon build command. You should see messages indicating that the isaac_ros_nitros package is being used:

--- stderr: zed_components
 * ISAAC ROS Nitros transport is available
---

Validate the installation #

To validate that the installation was successful and that the ZED ROS 2 Wrapper is properly configured with NVIDIA® Isaac™ ROS support, you can run the following command:

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=<your_camera_model> enable_ipc:=false

If the node starts successfully without errors, your installation is likely correct. You can also check the ROS 2 topic list to see if the ZED camera topics are being published:

ros2 topic list | grep zed

You should see topics related to the ZED camera, such as /zed/zed_node/rgb/image_rect_color/nitros and /zed/zed_node/depth/depth_registered/nitros, indicating that the ZED ROS 2 Wrapper is running with NITROS support as expected.

📌 Note: When using the ZED ROS 2 Wrapper with NVIDIA® Isaac™ ROS, make sure to set the enable_ipc parameter to false in the launch file to avoid potential conflicts with the NITROS transport. Read more in the ZED Isaac™ ROS Troubleshooting Guide and the NVIDIA® Isaac™ ROS Troubleshooting Documentation.