Coordinate Frames

Motion is relative to an observing position. For example, if a user is holding and walking with a ZED, the camera will not be in motion relative to the user, but it will be in motion relative to the floor.

In the same way, the ZED needs to express its motion information relative to a reference frame. This reference frame can either be the World Frame or the previous Camera Frame.

Camera Frame

The Camera Frame is the frame attached to the camera. Located at the back of the left eye of the device, it represents the viewport of the camera and allows to express relative pose.

To get the movement between the current camera position and the one corresponding to the previous image, use

getPosition(zed_pose, REFERENCE_FRAME::CAMERA).

World Frame

Describing the position of the camera in absolute real-world space requires a reference point that remains stationary as the device moves.

This global reference frame is called World Frame.

If no initial parameters are set, the World Frame is located at the place where the ZED first started motion tracking, and is oriented in the direction where the device was looking.

To get the position of the camera in real-world space, use

getPosition(zed_pose, REFERENCE_FRAME::WORLD).

World Frame Transform

You can change the initial location of the Camera Frame relative to the World Frame:

• By setting an initial world transform in PositionalTrackingParameters, the Camera Frame will start at the given position and orientation in world space. This is useful when using the ZED with an IMU so that the two sensors provide pose data relative to a common reference frame.
• If Spatial Memory is enabled and an Area file is loaded, the World Frame will be the one stored in the Area file.

1
sl::Transform initial_position;
2
// Set the initial position of the Camera Frame at 1m80 above the World Frame
3
initial_position.setTranslation(sl::Translation(0, 180, 0));
4
tracking_parameters.initial_world_transform = initial_position;

Frame Transforms

When the ZED is moved, the motion of the left eye and the center of the camera is not the same. To describe the actual motion of the center of the camera, you need to use a frame transform. A frame transform expresses the rotational and translational offsets between coordinate frames.

The function getPosition() returns the pose of the Camera Frame located on the left eye. To get the movement of the center of the camera, you must apply a rigid transform.

How to transform pose data

To get pose data in a user-specified coordinate frame:

• Measure the spatial transformation (translation and/or rotation) between the left eye of the camera and the local frame you want to get the pose data into.
• Get pose updates using getPosition().
• Transform each pose into the new coordinate frame using the translation and orientation offsets measured initially.
• Use the transformed pose in your application.

For example, to get the pose at the center of the camera, you need to translate the pose by half the baseline distance along the X axis:

1
void transformPose(sl::Transform &pose, float tx) {
2
    Transform transform_;
3
    // Translate the tracking frame by tx along the X axis  
4
    transform_.tx = tx;
5
    // Pose(new reference frame) = Pose (camera frame) * M, where M is the transform between two frames
6
    pose = pose * transform_;  
7
}
8
9
// Get the distance between the center of the camera and the left eye
10
float translation_left_to_center = zed.getCameraInformation().camera_configuration.calibration_parameters.T.x * 0.5f;
11
// Retrieve and transform the pose data into a new frame located at the center of the camera
12
POSITIONAL_TRACKING_STATE tracking_state = zed.getPosition(camera_pose, REFERENCE_FRAME::WORLD);
13
transformPose(camera_pose.pose_data, translation_left_to_center);

Frame transforms can be used to track the movement of different local components of a rigid body. For example, when using an HMD in VR applications, it is important to transform pose data at the pivot point of the user’s head and neck to accurately describe head movements. On a robot, a rigid spatial transformation between auxiliary sensors (IMU frame for example) and the Camera coordinate frame is necessary for sensor fusion.

Refer to the Positional Tracking sample for a code example of spatial transformation.

Selecting a Coordinate System

The ZED uses a three-dimensional cartesian coordinate system (X, Y, Z) to specify positions and orientations. The coordinate system can be either right-handed or left-handed.

By default, the ZED uses the Image Coordinate System that is right-handed with the positive Y-axis pointing down, X-axis pointing right, and Z-axis pointing away from the camera.

Different coordinate systems can be selected using sl::InitParameters:

• IMAGE - Right handed, y-down (default)
• LEFT_HANDED_Y_UP - Left handed, y-up (Unity 3D)
• RIGHT_HANDED_Y_UP - Right handed, y-up (OpenGL)
• LEFT_HANDED_Z_UP - Left handed, z-up (Unreal Engine)
• RIGHT_HANDED_Z_UP - Right handed, z-up (Cads, eg. 3DSMax)
• RIGHT_HANDED_Z_UP_X_FORWARD - Right handed, z-up, x-forward (ROS - REP 103)

1
init_params.coordinate_system = COORDINATE_SYSTEM::RIGHT_HANDED_Y_UP;

Coordinate Units

By default, coordinate values are in millimeters. It is possible to change coordinate units to meters, centimeters, feet and inches using:

1
init_params.coordinate_units = UNIT::METER; // Set units in meters