Fusion Class Reference

Holds Fusion process data and functions. More...

Functions

 Fusion ()
 Default constructor. More...
 
 ~Fusion ()
 Default destructor. More...
 
FUSION_ERROR_CODE init (sl::InitFusionParameters init_parameters=InitFusionParameters())
 Initialize the fusion module with the requested parameters. More...
 
void close ()
 Will deactivate all the fusion modules and free internal data. More...
 
FUSION_ERROR_CODE subscribe (CameraIdentifier uuid, CommunicationParameters param=CommunicationParameters(), sl::Transform pose=sl::Transform())
 Set the specified camera as a data provider. More...
 
FUSION_ERROR_CODE unsubscribe (CameraIdentifier uuid)
 Remove the specified camera from data provider. More...
 
FUSION_ERROR_CODE updatePose (CameraIdentifier uuid, sl::Transform pose)
 Updates the specified camera position inside fusion WORLD. More...
 
FUSION_ERROR_CODE getProcessMetrics (FusionMetrics &metrics)
 Get the metrics of the Fusion process, for the fused data as well as individual camera provider data. More...
 
std::map< CameraIdentifier, SENDER_ERROR_CODEgetSenderState ()
 Returns the state of each connected data senders. More...
 
FUSION_ERROR_CODE process ()
 Runs the main function of the Fusion, this trigger the retrieve and synchronization of all connected senders and updates the enabled modules. More...
 
FUSION_ERROR_CODE retrieveImage (sl::Mat &mat, CameraIdentifier uuid, sl::Resolution resolution=sl::Resolution(0, 0))
 Returns the current sl::VIEW::LEFT of the specified camera, the data is synchronized. More...
 
FUSION_ERROR_CODE retrieveMeasure (sl::Mat &mat, CameraIdentifier uuid, sl::MEASURE measure=sl::MEASURE::DEPTH, sl::Resolution resolution=sl::Resolution(0, 0))
 Returns the current measure of the specified camera, the data is synchronized. More...
 
FUSION_ERROR_CODE enableBodyTracking (BodyTrackingFusionParameters params=BodyTrackingFusionParameters())
 Enables the body tracking fusion module. More...
 
FUSION_ERROR_CODE retrieveBodies (sl::Bodies &objs, BodyTrackingFusionRuntimeParameters parameters=BodyTrackingFusionRuntimeParameters(), CameraIdentifier uuid=CameraIdentifier())
 Retrieves the body data, can be the fused data (default), or the raw data provided by a specific sender. More...
 
void disableBodyTracking ()
 Disable the body fusion tracking module. More...
 
FUSION_ERROR_CODE enablePositionalTracking (PositionalTrackingFusionParameters parameters=PositionalTrackingFusionParameters())
 Enables positional tracking fusion module. More...
 
POSITIONAL_TRACKING_STATE getPosition (Pose &camera_pose, sl::REFERENCE_FRAME reference_frame=REFERENCE_FRAME::WORLD, CameraIdentifier uuid=CameraIdentifier(), POSITION_TYPE position_type=POSITION_TYPE::FUSION)
 Get the Fused Position referenced to the first camera subscribed. If uuid is specified then project position on the referenced camera. More...
 
FUSION_ERROR_CODE ingestGNSSData (sl::GNSSData gnss_data)
 Ingest GNSS data from an external sensor into the fusion module. More...
 
POSITIONAL_TRACKING_STATE getCurrentGNSSData (sl::GNSSData &out)
 Returns the last synchronized gnss data. More...
 
GNSS_CALIBRATION_STATE getGeoPose (sl::GeoPose &pose)
 Returns the current GeoPose. More...
 
GNSS_CALIBRATION_STATE Geo2Camera (sl::LatLng &in, sl::Pose &out)
 Convert latitude / longitude into position in sl::Fusion coordinate system. More...
 
GNSS_CALIBRATION_STATE Camera2Geo (sl::Pose const &in, sl::GeoPose &out)
 Convert a position in sl::Fusion coordinate system in global world coordinate. More...
 
void disablePositionalTracking ()
 Disable the fusion positional tracking module. More...
 
sl::Timestamp getCurrentTimeStamp ()
 Return the current fusion timestamp, aligned with the synchronized GNSS and camera data. More...
 
GNSS_CALIBRATION_STATE getCurrentGNSSCalibrationSTD (float &yaw_std, sl::float3 &position_std)
 Get the current calibration uncertainty obtained during calibration process. More...
 
sl::Transform getGeoTrackingCalibration ()
 Get the calibration found between VIO and GNSS. More...
 
FUSION_ERROR_CODE enableSpatialMapping (SpatialMappingFusionParameters parameters=SpatialMappingFusionParameters())
 Initializes and starts the spatial mapping processes. More...
 
void requestSpatialMapAsync ()
 Starts the spatial map generation process in a non blocking thread from the spatial mapping process. More...
 
FUSION_ERROR_CODE getSpatialMapRequestStatusAsync ()
 Returns the spatial map generation status. This status allows to know if the mesh can be retrieved by calling retrieveSpatialMapAsync. More...
 
FUSION_ERROR_CODE retrieveSpatialMapAsync (Mesh &mesh)
 Retrieves the current generated spatial map only if SpatialMappingParameters::map_type was set as SPATIAL_MAP_TYPE::MESH. More...
 
FUSION_ERROR_CODE retrieveSpatialMapAsync (FusedPointCloud &fpc)
 Retrieves the current generated spatial map only if SpatialMappingParameters::map_type was set as SPATIAL_MAP_TYPE::FUSED_POINT_CLOUD. After calling requestSpatialMapAsync , this function allows you to retrieve the generated fused point cloud. The fused point cloud will only be available when getMeshRequestStatusAsync() returns SUCCESS. More...
 
void disableSpatialMapping ()
 Disables the spatial mapping process. More...
 

Detailed Description

Holds Fusion process data and functions.

Constructor and Destructor

◆ Fusion()

Fusion ( )

Default constructor.

◆ ~Fusion()

~Fusion ( )

Default destructor.

Functions

◆ init()

Initialize the fusion module with the requested parameters.

Parameters
init_parametersInitialization parameters.
Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ close()

void close ( )

Will deactivate all the fusion modules and free internal data.

◆ subscribe()

Set the specified camera as a data provider.

Parameters
uuidThe requested camera identifier.
paramThe communication parameters to connect to the camera.
poseThe World position of the camera, regarding the other camera of the setup.
Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ unsubscribe()

FUSION_ERROR_CODE unsubscribe ( CameraIdentifier  uuid)

Remove the specified camera from data provider.

Parameters
uuidThe requested camera identifier..
Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ updatePose()

FUSION_ERROR_CODE updatePose ( CameraIdentifier  uuid,
sl::Transform  pose 
)

Updates the specified camera position inside fusion WORLD.

Parameters
uuidThe requested camera identifier.
poseThe World position of the camera, regarding the other camera of the setup.
Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ getProcessMetrics()

FUSION_ERROR_CODE getProcessMetrics ( FusionMetrics metrics)

Get the metrics of the Fusion process, for the fused data as well as individual camera provider data.

Parameters
metricsThe process metrics.
Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ getSenderState()

std::map<CameraIdentifier, SENDER_ERROR_CODE> getSenderState ( )

Returns the state of each connected data senders.

Returns
The individual state of each connected senders.

◆ process()

FUSION_ERROR_CODE process ( )

Runs the main function of the Fusion, this trigger the retrieve and synchronization of all connected senders and updates the enabled modules.

Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ retrieveImage()

FUSION_ERROR_CODE retrieveImage ( sl::Mat mat,
CameraIdentifier  uuid,
sl::Resolution  resolution = sl::Resolution(0, 0) 
)

Returns the current sl::VIEW::LEFT of the specified camera, the data is synchronized.

Parameters
matthe CPU BGRA image of the requested camera.
uuidthe requested camera identifier.
resolutionthe requested resolution of the output image, can be lower or equal (default) to the original image resolution.
Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.
Note
Only the Left BGRA image is available.

◆ retrieveMeasure()

FUSION_ERROR_CODE retrieveMeasure ( sl::Mat mat,
CameraIdentifier  uuid,
sl::MEASURE  measure = sl::MEASURE::DEPTH,
sl::Resolution  resolution = sl::Resolution(0, 0) 
)

Returns the current measure of the specified camera, the data is synchronized.

Parameters
matthe CPU data of the requested camera.
uuidthe requested camera identifier.
measurethe requested measure type, by default DEPTH (F32_C1)
resolutionthe requested resolution of the output image, can be lower or equal (default) to the original image resolution.
Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.
Note
Only MEASURE: DEPTH, XYZ, XYZRGBA, XYZBGRA, XYZARGB, XYZABGR, DEPTH_U16_MM are available.

◆ enableBodyTracking()

Enables the body tracking fusion module.

Parameters
paramsStructure containing all specific parameters for body tracking fusion.
For more information, see the BodyTrackingFusionParameters documentation.
Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ retrieveBodies()

Retrieves the body data, can be the fused data (default), or the raw data provided by a specific sender.

Parameters
objsThe fused bodies will be saved into this objects.
parametersBody detection runtime settings, can be changed at each detection.
uuidIf set to a sender serial number (different from 0), this will retrieve the raw data provided by this sender.
Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ disableBodyTracking()

void disableBodyTracking ( )

Disable the body fusion tracking module.

◆ enablePositionalTracking()

Enables positional tracking fusion module.

Parameters
parametersA structure containing all the PositionalTrackingFusionParameters that define positional tracking fusion module.
Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ getPosition()

POSITIONAL_TRACKING_STATE getPosition ( Pose camera_pose,
sl::REFERENCE_FRAME  reference_frame = REFERENCE_FRAME::WORLD,
CameraIdentifier  uuid = CameraIdentifier(),
POSITION_TYPE  position_type = POSITION_TYPE::FUSION 
)

Get the Fused Position referenced to the first camera subscribed. If uuid is specified then project position on the referenced camera.

Parameters
camera_poseWill contain the fused position referenced by default in world (world is given by the calibration of the cameras system).
reference_frameDefines the reference from which you want the pose to be expressed. Default : REFERENCE_FRAME::WORLD.
uuidIf set to a sender serial number (different from 0), this will retrieve position projected on the requested camera if position_type is equal to POSITION_TYPE::FUSION or raw sender position if position_type is equal to POSITION_TYPE::RAW.
position_typeSelect if the position should the fused position re-projected in the camera with uuid or if the position should be the raw position (without fusion) of camera with uui.
Returns
POSITIONAL_TRACKING_STATE is the current state of the tracking process.

◆ ingestGNSSData()

FUSION_ERROR_CODE ingestGNSSData ( sl::GNSSData  gnss_data)

Ingest GNSS data from an external sensor into the fusion module.

Parameters
gnss_dataThe current GNSS data to combine with the current positional tracking data.
Returns
SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ getCurrentGNSSData()

POSITIONAL_TRACKING_STATE getCurrentGNSSData ( sl::GNSSData out)

Returns the last synchronized gnss data.

Parameters
outLast synchronized gnss data.
Returns
POSITIONAL_TRACKING_STATE is the current state of the tracking process.

◆ getGeoPose()

GNSS_CALIBRATION_STATE getGeoPose ( sl::GeoPose pose)

Returns the current GeoPose.

Parameters
poseThe current GeoPose.
Returns
GNSS_CALIBRATION_STATE is the current state of the tracking process.

◆ Geo2Camera()

GNSS_CALIBRATION_STATE Geo2Camera ( sl::LatLng in,
sl::Pose out 
)

Convert latitude / longitude into position in sl::Fusion coordinate system.

Parameters
inThe latitude / longitude to be converted in sl::Fusion coordinate system.
outConverted position in sl::Fusion coordinate system.
Returns
GNSS_CALIBRATION_STATE is the current state of the tracking process.

◆ Camera2Geo()

GNSS_CALIBRATION_STATE Camera2Geo ( sl::Pose const &  in,
sl::GeoPose out 
)

Convert a position in sl::Fusion coordinate system in global world coordinate.

Parameters
posePosition to convert in global world coordinate.
outConverted position in global world coordinate.
Returns
GNSS_CALIBRATION_STATE is the current state of the tracking process.

◆ disablePositionalTracking()

void disablePositionalTracking ( )

Disable the fusion positional tracking module.

◆ getCurrentTimeStamp()

sl::Timestamp getCurrentTimeStamp ( )

Return the current fusion timestamp, aligned with the synchronized GNSS and camera data.

Returns
sl::Timestamp current fusion timestamp.

◆ getCurrentGNSSCalibrationSTD()

GNSS_CALIBRATION_STATE getCurrentGNSSCalibrationSTD ( float &  yaw_std,
sl::float3 position_std 
)

Get the current calibration uncertainty obtained during calibration process.

Parameters
yaw_stdOutput yaw uncertainty.
x_stdOutput position uncertainty.
Returns
sl::GNSS_CALIBRATION_STATE representing current initialisation status.

◆ getGeoTrackingCalibration()

sl::Transform getGeoTrackingCalibration ( )

Get the calibration found between VIO and GNSS.

Returns
sl::Transform is the calibration found between VIO and GNSS during calibration process.

◆ enableSpatialMapping()

Initializes and starts the spatial mapping processes.

The spatial mapping will create a geometric representation of the scene based on both tracking data and 3D point clouds.
The resulting output can be a Mesh or a FusedPointCloud. It can be be obtained by calling retrieveSpatialMapAsync(). Note that retrieveSpatialMapAsync() should be called after requestSpatialMapAsync().

Parameters
spatial_mapping_parametersThe structure containing all the specific parameters for the spatial mapping.
Default: a balanced parameter preset between geometric fidelity and output file size. For more information, see the SpatialMappingParameters documentation.
Returns
SUCCESS if everything went fine, ERROR_CODE::FAILURE otherwise.
Note
The tracking (enablePositionalTracking()) needs to be enabled to use the spatial mapping.
Lower SpatialMappingParameters.range_meter and SpatialMappingParameters.resolution_meter for higher performance.
Warning
This fuction is only available for INTRA_PROCESS communication type.

◆ requestSpatialMapAsync()

void requestSpatialMapAsync ( )

Starts the spatial map generation process in a non blocking thread from the spatial mapping process.

The spatial map generation can take a long time depending on the mapping resolution and covered area. This function will trigger the generation of a mesh without blocking the program. You can get info about the current generation using getSpatialMapRequestStatusAsync(), and retrieve the mesh using retrieveSpatialMapAsync(...) .

Note
Only one mesh can be generated at a time. If the previous mesh generation is not over, new calls of the function will be ignored.

◆ getSpatialMapRequestStatusAsync()

FUSION_ERROR_CODE getSpatialMapRequestStatusAsync ( )

Returns the spatial map generation status. This status allows to know if the mesh can be retrieved by calling retrieveSpatialMapAsync.

Returns
SUCCESS if the mesh is ready and not yet retrieved, otherwise ERROR_CODE::FAILURE.


See requestSpatialMapAsync() for an example.

◆ retrieveSpatialMapAsync() [1/2]

FUSION_ERROR_CODE retrieveSpatialMapAsync ( Mesh mesh)

Retrieves the current generated spatial map only if SpatialMappingParameters::map_type was set as SPATIAL_MAP_TYPE::MESH.

After calling requestSpatialMapAsync , this function allows you to retrieve the generated mesh. The mesh will only be available when getMeshRequestStatusAsync() returns SUCCESS.

Parameters
mesh: The mesh to be filled with the generated spatial map.
Returns
SUCCESS if the mesh is retrieved, otherwise ERROR_CODE::FAILURE.
Note
This function only updates the necessary chunks and adds the new ones in order to improve update speed.
Warning
You should not modify the mesh between two calls of this function, otherwise it can lead to corrupted mesh.
If the SpatialMappingParameters::map_type has not been setup as SPATIAL_MAP_TYPE::MESH, the object will be empty.
See requestSpatialMapAsync() for an example.

◆ retrieveSpatialMapAsync() [2/2]

FUSION_ERROR_CODE retrieveSpatialMapAsync ( FusedPointCloud fpc)

Retrieves the current generated spatial map only if SpatialMappingParameters::map_type was set as SPATIAL_MAP_TYPE::FUSED_POINT_CLOUD. After calling requestSpatialMapAsync , this function allows you to retrieve the generated fused point cloud. The fused point cloud will only be available when getMeshRequestStatusAsync() returns SUCCESS.

Parameters
fpc: The fused point cloud to be filled with the generated spatial map.
Returns
SUCCESS if the fused point cloud is retrieved, otherwise ERROR_CODE::FAILURE.
Note
This function only updates the necessary chunks and adds the new ones in order to improve update speed.
Warning
You should not modify the fused point cloud between two calls of this function, otherwise it can lead to a corrupted fused point cloud.
If the SpatialMappingParameters::map_type has not been setup as SPATIAL_MAP_TYPE::FUSED_POINT_CLOUD, the object will be empty.


See requestSpatialMapAsync() for an example.

◆ disableSpatialMapping()

void disableSpatialMapping ( )

Disables the spatial mapping process.

The spatial mapping is immediately stopped.
If the mapping has been enabled, this function will automatically be called by close().

Note
This function frees the memory allocated for the spatial mapping, consequently, the spatial map cannot be retrieved after this call.