How to Calibrate your ZED camera with OpenCV

Even though the ZED cameras are factory calibrated, you may want to perform your own calibration and use its results in the ZED SDK. This can be useful for a specific use case or environment, such as when using the cameras underwater or behind a glass window.

This feature was introduced in ZED SDK version 3.4.

📌 Note: Using your own calibration will not erase the factory calibration; it will just replace it at runtime if requested using the API.

To enable this behavior, you have to specify an OpenCV calibration file as sl::InitParameters::optional_opencv_calibration_file [C++/Python].

Use the GitHub Open Source Calibration Tool #

You can use your own OpenCV calibration software if you want to create your own calibration file, or you can leverage the open source tool that we provide via GitHub: ZED OpenCV Calibration Tool.

Build the tool #

Open a terminal on your Linux system and execute the following commands:

git clone https://github.com/stereolabs/zed-opencv-calibration.git
cd zed-opencv-calibration

# Build the stereo calibration tool and the reprojection viewer
mkdir build && cd build
cmake ..
make -j$(nproc)

Perform the Stereo Calibration #

The Stereo Calibration Tool enables precise calibration of ZED stereo cameras and custom stereo rigs (e.g., two ZED X One cameras) using a checkerboard pattern. This process computes intrinsic camera parameters (focal length, principal point, distortion coefficients) and extrinsic parameters (relative position and orientation between cameras).

Checkerboard Pattern Requirements #

The calibration requires a printed checkerboard pattern with:

Important: The pattern dimensions refer to the number of inner corners (where black and white squares meet), not the number of squares.

Prepare the Calibration Target #
  • Print the checkerboard pattern maximized and attach it to a rigid, flat surface.
  • Ensure the pattern is perfectly flat and well-lit.
  • Avoid reflections or glare on the checkerboard surface.
Run the Calibration #

Default command to start calibration:

cd build/stereo_calibration/
./zed_stereo_calibration

This command tries to open the first connected ZED camera for live calibration using the default checkerboard settings.

You can also specify different options to calibrate virtual stereo cameras or use custom checkerboard parameters:


Usage: ./zed_stereo_calibration [options]
  --h_edges <value>      Number of horizontal inner edges of the checkerboard
  --v_edges <value>      Number of vertical inner edges of the checkerboard
  --square_size <value>  Size of a square in the checkerboard (in mm)
  --svo <file>           Path to the SVO file.
  --fisheye              Use fisheye lens model.
  --virtual              Use ZED X One cameras as a virtual stereo pair.
  --left_id <id>         Id of the left camera if using virtual stereo.
  --right_id <id>        Id of the right camera if using virtual stereo.
  --left_sn <sn>         S/N of the left camera if using virtual stereo.
  --right_sn <sn>        S/N of the right camera if using virtual stereo.
  --help, -h             Show this help message.
Stereo Calibration Example Commands #

  • ZED Stereo Camera using an SVO file:

    ./zed_stereo_calibration --svo <full_path_to_svo_file>

  • ZED Stereo Camera using a custom checkerboard (size 10x7 with 20mm squares):

    ./zed_stereo_calibration --h_edges 10 --v_edges 7 --square_size 20.0

  • Virtual Stereo Camera using camera IDs:

    ./zed_stereo_calibration --virtual --left_id 0 --right_id 1

  • Virtual Stereo Camera using camera serial numbers and a custom checkerboard (size 12x9 with 30mm squares):

    ./zed_stereo_calibration --virtual --left_sn <serial_number> --right_sn <serial_number> --h_edges 12 --v_edges 9 --square_size 30.0

  • Virtual Stereo Camera with fisheye lenses using camera serial numbers:

    ./zed_stereo_calibration --fisheye --virtual --left_sn <serial_number> --right_sn <serial_number>

📌 Note: You can easily obtain the serial numbers or the IDs of your connected ZED cameras by running the following command:

ZED_Explorer --all
The Calibration Process #

The calibration process consists of two main phases:

  1. Data Acquisition: Move the checkerboard in front of the camera(s) to capture diverse views. The tool provides real-time feedback on the quality of the captured data.
  2. Calibration Computation: Once sufficient data is collected, the tool computes the calibration parameters and saves them to two files.

The Data Acquisition phase consists of moving the checkerboard in front of the camera(s) to capture diverse views. The tool provides real-time feedback on the quality of the captured data regarding XY coverage, distance variation, and skewness.

When the checkerboard is placed in a position that you want to capture, press the Spacebar or the S key to capture the images.

  • If the checkerboard is detected in both images, and the captured data is different enough from the previously captured images, the data is accepted, and the quality indicators are updated.
  • If the data is not accepted, a message is displayed in the GUI output indicating the reason (e.g., checkerboard not detected, insufficient variation, etc.).

The blue dots that appear on the left image indicate the center of each checkerboard that has been detected and accepted so far. The size of the dots indicates the relative size of the checkerboard in the image (bigger dots mean closer to the camera).

In order to collect good calibration data, ensure that:

  • The checkerboard is always fully visible in both left and right images. Corners detected in both images are highlighted with colored visual markers.
  • The checkerboard moves over a wide area of the image frame. “Green” polygons appear on the left image to indicate the covered areas. When one of the 4 zones of the left image becomes fully green, the coverage requirement is met for that part of the image.
  • Red areas on the side of the left frame indicate zones that are not yet covered by the checkerboard. Try to make them as small as possible.
  • The checkerboard is moved closer and farther from the camera to ensure depth variation. At least one image covering almost the full left frame is required.
  • The checkerboard is tilted and rotated to provide different angles.

The “X”, “Y”, “Size”, and “Skew” percentages indicate the quality of the collected data for each criterion.

For X and Y, the minimum and maximum values correspond to the minimum and maximum position of the corner of the checkerboard closest to the image border. The COVERAGE indicates the size of the horizontal and vertical area covered by the checkerboard corners in the left image. The higher the coverage, the more the image is covered.

For Size, the minimum and maximum values correspond to the smallest and largest size of the checkerboard in the left image. The COVERAGE indicates the range of sizes of the checkerboard in the collected samples. A higher coverage means that the checkerboard was captured at a wider range of distances from the camera.

For Skew, the minimum and maximum values correspond to the minimum and maximum skewness angle of the checkerboard in the left image. The COVERAGE indicates the range of skew angles of the checkerboard in the collected samples. A higher coverage means that the checkerboard was captured at a wider range of angles. A value of 0° means the checkerboard is perfectly fronto-parallel to the camera; a theoretical maximum of 90° means the checkerboard is seen edge-on. Normally, the maximum achievable skew is around 40°.

If you cannot reach 100% for one of the metrics, ensure that it is as high as possible, and move the checkerboard to different positions to maximize the coverage by acquiring the maximum number of samples.

Here are some tips to improve each metric:

  • To raise the “X” and “Y” metrics, move the checkerboard to the edges and corners of the left image while keeping it fully visible in the right frame.
  • To raise the “Size” metric, move the checkerboard closer and farther from the camera. You must acquire at least one image where the checkerboard is covering almost the full left image and one where it is smaller and corners are barely detected [see the image below].
  • To raise the “Skew” metric, rotate the checkerboard at different angles. It is easier to obtain different skew values if the checkerboard is closer to the camera and rotated around the vertical and horizontal axes simultaneously.

The “Calibrate” process will automatically start when either of these conditions is met:

  • All metrics reach 100% and the minimum number of samples is collected.
  • The maximum number of samples is reached (even if not all metrics reach 100%).

For each metric, the GUI shows the following information in a table:

  • MIN_VAL: Minimum value stored in all the samples collected so far.
  • MAX_VAL: Maximum value stored in all the samples collected so far.
  • COVERAGE: The difference between the MIN_VAL and MAX_VAL, representing the range of variation in the collected samples.
  • REQUIRED: The minimum required value for the COVERAGE to consider the metric as satisfied.
  • SCORE: The percentage score for the metric, calculated as (COVERAGE / REQUIRED) * 100%.

You can follow the steps of the calibration process in the terminal output:

  1. The left camera is calibrated first, followed by the right camera to obtain the intrinsic parameters.
  2. Finally, the stereo calibration is performed to compute the extrinsic parameters between the two cameras.

Good calibration results typically yield a reprojection error below 0.5 pixels for each calibration step.

If any reprojection error is too high, the calibration is not accurate enough and should be redone. Before recalibrating, verify the following:

  • The checkerboard is perfectly flat and securely mounted.
  • The checkerboard is well-lit with even, stable lighting.
  • Camera lenses are clean and free of smudges or dust.
  • No reflections or glare appear on the checkerboard surface.

After a good calibration is complete, two files are generated:

  • zed_calibration_<serial_number>.yml: Contains intrinsic and extrinsic parameters for the stereo camera setup in OpenCV format.
  • SN<serial_number>.conf: Contains the calibration parameters in ZED SDK format.

You can use these files in your ZED SDK applications:

📌 Note: When calibrating a virtual ZED X One stereo rig, the serial number of the Virtual Stereo Camera is generated by the ZED SDK using the serial numbers of the two individual cameras. Make sure to use this generated serial number when loading the calibration in your application to have a unique identifier for the virtual stereo setup.

Use custom OpenCV code #

When using custom OpenCV calibration code, make sure to save the calibration results in a format compatible with the ZED SDK:

📌 Note: For better results, perform your acquisition in FHD or 2K resolution. The ZED SDK will automatically adapt the given calibration file to the requested camera resolution at runtime.

You can refer to the ZED OpenCV Calibration Tool GitHub repository for an example of C++ code to export a valid file.

Here is an example of a valid OpenCV calibration file for a virtual ZED stereo camera with a baseline of ~10 cm:

%YAML:1.0
---
Size: [ 1920, 1200 ]
K_LEFT: !!opencv-matrix
     rows: 3
     cols: 3
     dt: d
     data: [ 7.2820438736188862e+02, 0., 9.5834115836072010e+02, 0.,
             7.4158763721083221e+02, 5.3427071881183190e+02, 0., 0., 1. ]
K_RIGHT: !!opencv-matrix
     rows: 3
     cols: 3
     dt: d
     data: [ 7.2958238593130909e+02, 0., 9.0162007159649295e+02, 0.,
             7.4230255873089459e+02, 6.7036040993040706e+02, 0., 0., 1. ]
D_LEFT: !!opencv-matrix
     rows: 1
     cols: 14
     dt: d
     data: [ 1.1750221618870700e-01, -5.5471356596234270e-02,
             -4.0951672660553275e-04, 4.3039096524305953e-04,
             -8.1558279494929956e-02, 1.0798893738294896e-01,
             1.8282436403220467e-02, -1.1198329799056186e-01, 0., 0., 0., 0.,
             0., 0. ]
D_RIGHT: !!opencv-matrix
     rows: 1
     cols: 14
     dt: d
     data: [ -1.0478075413266838e-01, -5.1092750848071800e-02,
             6.1548787402378828e-05, 4.3660163135302086e-04,
             -1.1350089318741453e-01, -1.0953227410609423e-01,
             1.5775967171221997e-02, -1.4941057596185622e-01, 0., 0., 0., 0.,
             0., 0. ]
R: !!opencv-matrix
     rows: 3
     cols: 1
     dt: d
     data: [ 9.7165302011531611e-03, 6.6233703125605729e-03,
             1.5351116217350151e-02 ]
T: !!opencv-matrix
     rows: 3
     cols: 1
     dt: d
     data: [ -9.9429145000373040e+01, -5.2023685972818123e-01,
             9.5608016907871174e-01 ]

Make sure to replace the intrinsic (K_LEFT, K_RIGHT, D_LEFT, D_RIGHT) and extrinsic (R, T) parameters with those obtained from your calibration process.

The rotation information R is represented as a rotation vector (Rodrigues format), and the translation information T is represented in millimeters.

If required, you can use the cv::Rodrigues function to convert between rotation matrix and rotation vector formats. Here’s a working example.

📌 Note: The first component of the translation vector T represents the baseline distance between the two cameras and should be a negative value (e.g., -100.0 for a 10 cm baseline). This is a good indicator of a correct calibration: if the value is close to the expected baseline of your stereo rig, then the calibration is likely accurate.