Getting Started with GStreamer and ZED

GStreamer is a popular framework used to create custom media pipelines by combining modular plugins. The ZED GStreamer plugins greatly simplifies the use of ZED camera and SDK in a GStreamer media pipeline.

Key Features

The ZED GStreamer Plugins package is composed of several modules:

  • zedsrc: acquires camera color image, depth map, sensors data and object detection information and pushes them in a GStreamer pipeline.
  • zedmeta: GStreamer library to define and handle the ZED metadata (Positional Tracking data, Sensors data, Detected Object data, Detected Skeletons data).
  • zeddemux: receives a composite zedsrc stream (color left + color right + metadata or color left + depth map + metadata), processes the eventual depth data and pushes them in two separated new streams named src_left and src_aux. A third source pad is optionally created for metadata to be externally processed.
  • zeddatamux: receives a video stream compatible with ZED caps and a ZED Data Stream generated by the zeddemux and adds metadata to the video stream. This is useful if metadata are removed by a filter that does not automatically propagate them.
  • zeddatacsvsink: example sink plugin that receives ZED metadata, extracts the Positional Tracking and the Sensors Data information and save them in a CSV file.
  • zedodoverlay: example transform filter plugin that receives ZED combined stream with metadata, extracts object detection information and draws the overlays on the oncoming image frames.
  • RTSP Server: application for Linux that instantiates an RTSP server from a text launch pipeline “gst-launch” like.

Installation

Prerequisites

Windows installation

  • Install the latest ZED SDK from the official download page [Optional to compile the zedsrc plugin to acquire data from a ZED camera device]

  • Install Git

  • Install CMake

  • Install a GStreamer distribution (both runtime and development installers).

  • The installer should set the installation path via the GSTREAMER_1_0_ROOT_X86_64 environment variable.

  • Add the path %GSTREAMER_1_0_ROOT_X86_64%\bin to the system variable PATH

  • [Optional] Install OpenCV to build the zedodverlay filter

  • Run the following commands from a terminal or command prompt, assuming CMake and Git are in your PATH.

    git clone https://github.com/stereolabs/zed-gstreamer.git
    cd zed-gstreamer
    mkdir build
    cd build
    cmake -G "Visual Studio 16 2019" ..
    cmake --build . --target INSTALL --config Release
    

Linux installation

Install prerequisites

  • Install the latest ZED SDK from the official download page

  • Update list of apt available packages

    sudo apt update

  • Install GCC compiler and build tools

    sudo apt install build-essential

  • Install CMake build system

    sudo apt install cmake

  • Install GStreamer, the development packages and useful tools:

    sudo apt install libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav libgstrtspserver-1.0-0 gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstrtspserver-1.0-dev

  • [Optional] Install OpenCV to build the zedodverlay filter

    sudo apt install libopencv-dev libopencv-contrib-dev

Clone the repository

git clone https://github.com/stereolabs/zed-gstreamer.git
cd zed-gstreamer

Build

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make # Note: do not use the `-j` flag because parallel build is not supported
sudo make install

Installation test

Open a command line console and try the following commands to verify that everything is correctly installed:

  • Check ZED Video Source Plugin installation inspecting its properties:

    gst-inspect-1.0 zedsrc

  • Check ZED Video Demuxer installation inspecting its properties:

    gst-inspect-1.0 zeddemux

  • Check ZED Data Mux Plugin installation inspecting its properties:

    gst-inspect-1.0 zeddatamux

  • Check ZED CSV Sink Plugin installation inspecting its properties:

    `gst-inspect-1.0 zeddatacsvsink`
    
  • Check ZED Object Detection Overlay Plugin installation inspecting its properties:

    `gst-inspect-1.0 zedodoverlay`