Accessing Raw NV12 Buffers

This guide explains how to access raw NV12 image buffers directly from the ZED SDK using the RawBuffer API. This is an advanced, zero-copy interface designed for performance-critical applications.

This is an advanced low-level API. Improper use can crash the Argus stack responsible for camera operations or destabilize the system. Use this API only if you understand NVIDIA® multimedia buffer handling.

This API is only available starting from ZED SDK version 5.2. It is supported on NVIDIA® Jetson™ platforms with GMSL2 cameras. It is not available on x86 platforms, with USB cameras, or using SVO and Network streaming as input sources.

Overview

The RawBuffer API provides direct access to native NvBufSurface buffers from the ZED X One camera capture pipeline. Instead of copying image data, this API gives you a pointer to the underlying buffer, enabling zero-copy integration with NVIDIA multimedia frameworks.

Use Cases

This API is particularly useful for:

GStreamer pipelines – Feed camera frames directly into GStreamer without memory copies
NVIDIA DeepStream – Low-latency inference pipelines for real-time object detection
Custom CUDA processing – Direct GPU access to camera buffers
Hardware video encoding – Direct integration with NVENC

Enabling the API

The RawBuffer API is disabled by default. To enable it, define SL_ENABLE_ADVANCED_CAPTURE_API before including the ZED SDK header:

C++

1 #define SL_ENABLE_ADVANCED_CAPTURE_API
2 #include <sl/CameraOne.hpp>

Code Example

C++

1 #define SL_ENABLE_ADVANCED_CAPTURE_API
2 #include <sl/CameraOne.hpp>
3 #include <nvbufsurface.h>  // NVIDIA buffer surface API
4 
5 using namespace sl;
6 
7 int main() {
8     CameraOne zed;
9     InitParametersOne init_params;
10     init_params.camera_resolution = RESOLUTION::HD1080;
11     init_params.camera_fps = 30;
12 
13     if (zed.open(init_params) != ERROR_CODE::SUCCESS) {
14         return -1;
15     }
16 
17     while (true) {
18         if (zed.grab() == ERROR_CODE::SUCCESS) {
19             RawBuffer raw;
20             if (zed.retrieveImage(raw) == ERROR_CODE::SUCCESS) {
21                 if (raw.isValid()) {
22                     // Get NvBufSurface pointers (zero-copy)
23                     void* nvbuf = raw.getRawBuffer();
24 
25                     // Cast to NvBufSurface* for use with NVIDIA APIs
26                     NvBufSurface* surf = static_cast<NvBufSurface*>(nvbuf);
27 
28                     // Access buffer properties
29                     uint64_t timestamp = raw.getTimestamp();
30                     unsigned int width = raw.getWidth();
31                     unsigned int height = raw.getHeight();
32 
33                     // Process the NV12 buffer with your pipeline...
34                     // Example: feed to GStreamer, DeepStream, or custom CUDA kernel
35 
36                     // Buffer is automatically released when 'raw' goes out of scope
37                 }
38             }
39         }
40     }
41 
42     zed.close();
43     return 0;
44 }

Integration with DeepStream

For NVIDIA® DeepStream integration, you can pass the NvBufSurface object directly to your inference pipeline:

C++

1 #define SL_ENABLE_ADVANCED_CAPTURE_API
2 #include <sl/Camera.hpp>
3 #include <nvbufsurface.h>
4 #include <nvds_meta.h>
5 
6 // In your DeepStream pipeline loop:
7 RawBuffer raw;
8 if (zed.retrieveImage(raw) == ERROR_CODE::SUCCESS && raw.isValid()) {
9     NvBufSurface* surface = static_cast<NvBufSurface*>(raw.getRawBuffer());
10     
11     // Feed surface to DeepStream batch
12     // surface->surfaceList[0] contains the NV12 frame data
13     
14     // Process with nvinfer, nvtracker, etc.
15 }

Important Warnings

DO NOT manually destroy the NvBufSurface (e.g., do not call NvBufSurfaceDestroy, NvBufSurfaceUnMap, etc.). The buffers are owned and managed by the SDK. Manual destruction will cause crashes or undefined behavior.

The SDK manages the buffer memory – you only have read access
Hold the RawBuffer for as short a time as possible to avoid blocking the capture pipeline
Currently, only RAW_BUFFER_TYPE::NVBUFSURFACE (NVIDIA® Argus native buffers) is supported
This API is available only on NVIDIA Jetson platforms with GMSL2 cameras

Buffer Format

The raw buffers contain NV12 formatted image data:

Y plane: Full-resolution luminance data
UV plane: Half-resolution (2x2 subsampled) chrominance data

This is the native capture format from the ZED X One sensor, providing maximum performance for pipelines that work with NV12 directly.