Fusion Class Reference

Holds Fusion process data and functions. More...

Functions
FUSION_ERROR_CODE	init (self, InitFusionParameters init_fusion_parameters)
	Initialize the fusion module with the requested parameters. More...
None	close (self)
	Will deactivate all the fusion modules and free internal data.
FUSION_ERROR_CODE	subscribe (self, CameraIdentifier uuid, CommunicationParameters communication_parameters, Transform pose)
	Set the specified camera as a data provider. More...
FUSION_ERROR_CODE	unsubscribe (self, CameraIdentifier uuid)
	Remove the specified camera from data provider. More...
FUSION_ERROR_CODE	update_pose (self, CameraIdentifier uuid, Transform pose)
	Updates the specified camera position inside fusion WORLD. More...
tuple(FUSION_ERROR_CODE, FusionMetrics)	get_process_metrics (self)
	Get the metrics of the Fusion process, for the fused data as well as individual camera provider data. More...
dict	get_sender_state (self)
	Returns the state of each connected data senders. More...
FUSION_ERROR_CODE	process (self)
	Runs the main function of the Fusion, this trigger the retrieve and synchronization of all connected senders and updates the enabled modules. More...
FUSION_ERROR_CODE	enable_body_tracking (self, BodyTrackingFusionParameters params)
	Enables the body tracking fusion module. More...
FUSION_ERROR_CODE	retrieve_bodies (self, Bodies bodies, BodyTrackingFusionRuntimeParameters parameters, CameraIdentifier uuid=CameraIdentifier(0), FUSION_REFERENCE_FRAME reference_frame=FUSION_REFERENCE_FRAME.BASELINK)
	Retrieves the body data, can be the fused data (default), or the raw data provided by a specific sender. More...
FUSION_ERROR_CODE	enable_object_detection (self, ObjectDetectionFusionParameters, params=ObjectDetectionFusionParameters())
	Enables the object detection fusion module. More...
FUSION_ERROR_CODE	retrieve_objects_all_od_groups (self, dict, objs, FUSION_REFERENCE_FRAME reference_frame=FUSION_REFERENCE_FRAME.BASELINK)
	Retrieves all the fused objects data. More...
FUSION_ERROR_CODE	retrieve_objects_one_od_group (self, Objects, objs, str, fused_od_group_name, FUSION_REFERENCE_FRAME reference_frame=FUSION_REFERENCE_FRAME.BASELINK)
	Retrieves the fused objects of a given fused OD group. More...
FUSION_ERROR_CODE	retrieve_raw_objects_all_ids (self, dict, objs, CameraIdentifier, uuid)
	Retrieves all the raw objects data provided by a specific sender. More...
FUSION_ERROR_CODE	retrieve_raw_objects_one_id (self, Objects, py_objects, CameraIdentifier, uuid, uint, instance_id)
	Retrieves the raw objects data provided by a specific sender and a specific instance id. More...
None	disable_objects_detection (self)
	Disable the body fusion tracking module.
FUSION_ERROR_CODE	retrieve_image (self, Mat, mat, CameraIdentifier, uuid, Resolution, resolution=Resolution(0, 0))
	Returns the current sl.VIEW.LEFT of the specified camera, the data is synchronized. More...
FUSION_ERROR_CODE	retrieve_measure (self, Mat, mat, CameraIdentifier, uuid, MEASURE measure, Resolution, resolution=Resolution(0, 0), FUSION_REFERENCE_FRAME reference_frame=FUSION_REFERENCE_FRAME.BASELINK)
	Returns the current measure of the specified camera, the data is synchronized. More...
None	disable_body_tracking (self)
	Disable the body fusion tracking module.
FUSION_ERROR_CODE	enable_positionnal_tracking (self, PositionalTrackingFusionParameters parameters)
	Enables positional tracking fusion module. More...
FUSION_ERROR_CODE	ingest_gnss_data (self, GNSSData gnss_data)
	Ingest GNSS data from an external sensor into the fusion module. More...
POSITIONAL_TRACKING_STATE	get_position (self, Pose camera_pose, REFERENCE_FRAME reference_frame=REFERENCE_FRAME.WORLD, CameraIdentifier uuid=CameraIdentifier(), POSITION_TYPE position_type=POSITION_TYPE.FUSION)
	Get the Fused Position referenced to the first camera subscribed. More...
POSITIONAL_TRACKING_STATE	get_current_gnss_data (self, GNSSData gnss_data)
	Returns the last synchronized gnss data. More...
GNSS_FUSION_STATUS	get_geo_pose (self, GeoPose pose)
	Returns the current GeoPose. More...
GNSS_FUSION_STATUS	geo_to_camera (self, LatLng input, Pose output)
	Convert latitude / longitude into position in sl::Fusion coordinate system. More...
GNSS_FUSION_STATUS	camera_to_geo (self, Pose input, GeoPose output)
	Convert a position in sl.Fusion coordinate system in global world coordinate. More...
Timestamp	get_current_timestamp (self)
	Return the current fusion timestamp, aligned with the synchronized GNSS and camera data. More...
None	disable_positionnal_tracking (self)
	Disable the fusion positional tracking module. More...
FUSION_ERROR_CODE	ENU_to_geo (self, ENU input, LatLng output)
	Convert ENU to LatLng. More...
FUSION_ERROR_CODE	geo_to_ENU (self, LatLng input, ENU out)
	Convert LatLng to ENU. More...
tuple[GNSS_FUSION_STATUS, float, np.array]	get_current_gnss_calibration_std (self)
	Get the current calibration uncertainty obtained during calibration process. More...
Transform	get_geo_tracking_calibration (self)
	Get the calibration found between VIO and GNSS. More...
FUSION_ERROR_CODE	enable_spatial_mapping (self, SpatialMappingFusionParameters, parameters)
	Initializes and starts the spatial mapping processes. More...
None	request_spatial_map_async (self)
	Starts the spatial map generation process in a non blocking thread from the spatial mapping process. More...
FUSION_ERROR_CODE	get_spatial_map_request_status_async (self)
	Returns the spatial map generation status. More...
FUSION_ERROR_CODE	retrieve_spatial_map_async (self, py_mesh)
	Retrieves the current generated spatial map. More...
None	disable_spatial_mapping (self)
	Disables the spatial mapping process. More...

Detailed Description

Holds Fusion process data and functions.

Functions

init()

FUSION_ERROR_CODE init	(		self,
		InitFusionParameters	init_fusion_parameters
	)

Initialize the fusion module with the requested parameters.

Parameters

init_parameters

Initialization parameters.

Returns

ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an ERROR_CODE.

Referenced by InitParameters.__cinit__(), InitParameters.async_grab_camera_recovery(), InitParametersOne.async_grab_camera_recovery(), InitParameters.async_image_retrieval(), InitParameters.camera_disable_self_calib(), InitParameters.camera_fps(), InitParametersOne.camera_fps(), InitParameters.camera_image_flip(), InitParameters.camera_resolution(), InitParametersOne.camera_resolution(), InitParameters.coordinate_system(), InitParametersOne.coordinate_system(), InitParameters.coordinate_units(), InitParametersOne.coordinate_units(), InitParameters.depth_maximum_distance(), InitParameters.depth_minimum_distance(), InitParameters.depth_mode(), InitParameters.depth_stabilization(), InitParametersOne.enable_hdr(), InitParameters.enable_image_enhancement(), InitParameters.enable_image_validity_check(), InitParameters.enable_right_side_measure(), InitParameters.grab_compute_capping_fps(), InitParameters.load(), InitParameters.maximum_working_resolution(), InitParameters.open_timeout_sec(), InitParameters.optional_opencv_calibration_file(), InitParameters.optional_settings_path(), InitParametersOne.optional_settings_path(), InitParameters.save(), InitParameters.sdk_gpu_id(), InitFusionParameters.sdk_gpu_id(), InitParameters.sdk_verbose(), InitParametersOne.sdk_verbose(), InitParameters.sdk_verbose_log_file(), InitParametersOne.sdk_verbose_log_file(), InitParameters.sensors_required(), InitParameters.set_from_camera_id(), InitParametersOne.set_from_camera_id(), InitParameters.set_from_serial_number(), InitParametersOne.set_from_serial_number(), InitParameters.set_from_stream(), InitParametersOne.set_from_stream(), InitParameters.set_from_svo_file(), InitParametersOne.set_from_svo_file(), InitParameters.svo_real_time_mode(), and InitParametersOne.svo_real_time_mode().

subscribe()

FUSION_ERROR_CODE subscribe	(		self,
		CameraIdentifier	uuid,
		CommunicationParameters	communication_parameters,
		Transform	pose
	)

Set the specified camera as a data provider.

Parameters

uuid	The requested camera identifier.
communication_parameters	The communication parameters to connect to the camera.
pose	The World position of the camera, regarding the other camera of the setup.

Returns

FUSION_ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

unsubscribe()

FUSION_ERROR_CODE unsubscribe	(		self,
		CameraIdentifier	uuid
	)

Remove the specified camera from data provider.

Parameters

uuid	The requested camera identifier.

Returns

FUSION_ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

update_pose()

FUSION_ERROR_CODE update_pose	(		self,
		CameraIdentifier	uuid,
		Transform	pose
	)

Updates the specified camera position inside fusion WORLD.

Parameters

uuid	The requested camera identifier.
pose	The World position of the camera, regarding the other camera of the setup.

Returns

FUSION_ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

get_process_metrics()

tuple(FUSION_ERROR_CODE, FusionMetrics) get_process_metrics

(

self

)

Get the metrics of the Fusion process, for the fused data as well as individual camera provider data.

Parameters

metrics

The process metrics.

Returns

FUSION_ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

The process metrics.

get_sender_state()

dict get_sender_state

(

self

)

Returns the state of each connected data senders.

Returns

The individual state of each connected senders.

process()

FUSION_ERROR_CODE process

(

self

)

Runs the main function of the Fusion, this trigger the retrieve and synchronization of all connected senders and updates the enabled modules.

Returns

FUSION_ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

enable_body_tracking()

FUSION_ERROR_CODE enable_body_tracking	(		self,
		BodyTrackingFusionParameters	params
	)

Enables the body tracking fusion module.

Parameters

params

Structure containing all specific parameters for body tracking fusion.

Returns

FUSION_ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

retrieve_bodies()

FUSION_ERROR_CODE retrieve_bodies	(		self,
		Bodies	bodies,
		BodyTrackingFusionRuntimeParameters	parameters,
		CameraIdentifier	uuid = CameraIdentifier(0),
		FUSION_REFERENCE_FRAME	reference_frame = FUSION_REFERENCE_FRAME.BASELINK
	)

Retrieves the body data, can be the fused data (default), or the raw data provided by a specific sender.

Parameters

bodies	The fused bodies will be saved into this objects.
parameters	Body detection runtime settings, can be changed at each detection.
uuid	The id of the sender.
reference_frame	The reference frame in which the objects will be expressed. Default: FUSION_REFERENCE_FRAME::BASELINK.

Returns

FUSION_ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

enable_object_detection()

FUSION_ERROR_CODE enable_object_detection	(		self,
			ObjectDetectionFusionParameters,
			params = ObjectDetectionFusionParameters()
	)

Enables the object detection fusion module.

Parameters

params

Structure containing all specific parameters for object detection fusion. For more information, see the ObjectDetectionFusionParameters documentation.

Returns

SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

retrieve_objects_all_od_groups()

FUSION_ERROR_CODE retrieve_objects_all_od_groups	(		self,
			dict,
			objs,
		FUSION_REFERENCE_FRAME	reference_frame = FUSION_REFERENCE_FRAME.BASELINK
	)

Retrieves all the fused objects data.

Parameters

objs	The fused objects will be saved into this dictionary of objects.
reference_frame	The reference frame in which the objects will be expressed. Default: FUSION_REFERENCE_FRAME::BASELINK.

Returns

SUCCESS if it goes as it should, otherwise it returns a FUSION_ERROR_CODE.

retrieve_objects_one_od_group()

FUSION_ERROR_CODE retrieve_objects_one_od_group	(		self,
			Objects,
			objs,
			str,
			fused_od_group_name,
		FUSION_REFERENCE_FRAME	reference_frame = FUSION_REFERENCE_FRAME.BASELINK
	)

Retrieves the fused objects of a given fused OD group.

Parameters

objs	The fused objects will be saved into this objects.
fused_od_group_name	The name of the fused objects group to retrieve.
reference_frame	The reference frame in which the objects will be expressed. Default: FUSION_REFERENCE_FRAME::BASELINK.

Returns

SUCCESS if it goes as it should, otherwise it returns a FUSION_ERROR_CODE.

retrieve_raw_objects_all_ids()

FUSION_ERROR_CODE retrieve_raw_objects_all_ids	(		self,
			dict,
			objs,
			CameraIdentifier,
			uuid
	)

Retrieves all the raw objects data provided by a specific sender.

Parameters

objs	The fused objects will be saved into this dictionary of objects.
uuid	Retrieve the raw data provided by this sender.

retrieve_raw_objects_one_id()

FUSION_ERROR_CODE retrieve_raw_objects_one_id	(		self,
			Objects,
			py_objects,
			CameraIdentifier,
			uuid,
			uint,
			instance_id
	)

Retrieves the raw objects data provided by a specific sender and a specific instance id.

Parameters

objs	The fused objects will be saved into this objects.
uuid	Retrieve the raw data provided by this sender.
instance_id	Retrieve the objects inferred by the model with this ID only.

Returns

SUCCESS if it goes as it should, otherwise it returns a FUSION_ERROR_CODE.

retrieve_image()

FUSION_ERROR_CODE retrieve_image	(		self,
			Mat,
			mat,
			CameraIdentifier,
			uuid,
			Resolution,
			resolution = Resolution(0,0)
	)

Returns the current sl.VIEW.LEFT of the specified camera, the data is synchronized.

Parameters

mat	the CPU BGRA image of the requested camera.
resolution	the requested resolution of the output image, can be lower or equal (default) to the original image resolution.
uuid	If set to a sender serial number (different from 0), this will retrieve the raw data provided by this sender.

Returns

FUSION_ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

retrieve_measure()

FUSION_ERROR_CODE retrieve_measure	(		self,
			Mat,
			mat,
			CameraIdentifier,
			uuid,
		MEASURE	measure,
			Resolution,
			resolution = Resolution(0,0),
		FUSION_REFERENCE_FRAME	reference_frame = FUSION_REFERENCE_FRAME.BASELINK
	)

Returns the current measure of the specified camera, the data is synchronized.

Parameters

mat	the CPU data of the requested camera.
uuid	The id of the sender.
measure	measure: the requested measure type, by default DEPTH (F32_C1).
resolution	the requested resolution of the output image, can be lower or equal (default) to the original image resolution.
reference_frame	The reference frame in which the objects will be expressed. Default: FUSION_REFERENCE_FRAME::BASELINK.

Returns

FUSION_ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

enable_positionnal_tracking()

FUSION_ERROR_CODE enable_positionnal_tracking	(		self,
		PositionalTrackingFusionParameters	parameters
	)

Enables positional tracking fusion module.

Parameters

parameters

A structure containing all the PositionalTrackingFusionParameters that define positional tracking fusion module.

Returns

FUSION_ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

ingest_gnss_data()

FUSION_ERROR_CODE ingest_gnss_data	(		self,
		GNSSData	gnss_data
	)

Ingest GNSS data from an external sensor into the fusion module.

Parameters

gnss_data

The current GNSS data to combine with the current positional tracking data.

Returns

FUSION_ERROR_CODE.SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

get_position()

POSITIONAL_TRACKING_STATE get_position	(		self,
		Pose	camera_pose,
		REFERENCE_FRAME	reference_frame = REFERENCE_FRAME.WORLD,
		CameraIdentifier	uuid = CameraIdentifier(),
		POSITION_TYPE	position_type = POSITION_TYPE.FUSION
	)

Get the Fused Position referenced to the first camera subscribed.

If uuid is specified then project position on the referenced camera.

Parameters

camera_pose	Will contain the fused position referenced by default in world (world is given by the calibration of the cameras system).
reference_frame	Defines the reference from which you want the pose to be expressed. Default : REFERENCE_FRAME.WORLD.
uuid	If set to a sender serial number (different from 0), this will retrieve position projected on the requested camera if position_type is equal to POSITION_TYPE.FUSION or raw sender position if position_type is equal to POSITION_TYPE.RAW.
position_type	Select if the position should the fused position re-projected in the camera with uuid or if the position should be the raw position (without fusion) of camera with uui.

Returns

POSITIONAL_TRACKING_STATE is the current state of the tracking process.

get_current_gnss_data()

POSITIONAL_TRACKING_STATE get_current_gnss_data	(		self,
		GNSSData	gnss_data
	)

Returns the last synchronized gnss data.

Parameters

out	[out]: Last synchronized gnss data.

Returns

POSITIONAL_TRACKING_STATE is the current state of the tracking process.

get_geo_pose()

GNSS_FUSION_STATUS get_geo_pose	(		self,
		GeoPose	pose
	)

Returns the current GeoPose.

Parameters

pose	[out]: The current GeoPose.

Returns

GNSS_FUSION_STATUS is the current state of the tracking process.

geo_to_camera()

GNSS_FUSION_STATUS geo_to_camera	(		self,
		LatLng	input,
		Pose	output
	)

Convert latitude / longitude into position in sl::Fusion coordinate system.

Parameters

input	[in]: The latitude / longitude to be converted in sl::Fusion coordinate system.
out	[out]: Converted position in sl.Fusion coordinate system.

Returns

GNSS_FUSION_STATUS is the current state of the tracking process.

camera_to_geo()

GNSS_FUSION_STATUS camera_to_geo	(		self,
		Pose	input,
		GeoPose	output
	)

Convert a position in sl.Fusion coordinate system in global world coordinate.

Parameters

pose	[in]: Position to convert in global world coordinate.
pose	[out]: Converted position in global world coordinate.

Returns

GNSS_FUSION_STATUS is the current state of the tracking process.

get_current_timestamp()

Timestamp get_current_timestamp

(

self

)

Return the current fusion timestamp, aligned with the synchronized GNSS and camera data.

Returns

current fusion timestamp.

disable_positionnal_tracking()

None disable_positionnal_tracking

(

self

)

Disable the fusion positional tracking module.

The positional tracking is immediately stopped. If a file path is given, saveAreaMap(area_file_path) will be called asynchronously. See getAreaExportState() to get the exportation state.

ENU_to_geo()

FUSION_ERROR_CODE ENU_to_geo	(		self,
		ENU	input,
		LatLng	output
	)

Convert ENU to LatLng.

Concert an ENU position into LatLng

geo_to_ENU()

FUSION_ERROR_CODE geo_to_ENU	(		self,
		LatLng	input,
		ENU	out
	)

Convert LatLng to ENU.

Convert am LatLng to ENU

get_current_gnss_calibration_std()

tuple[GNSS_FUSION_STATUS, float, np.array] get_current_gnss_calibration_std

(

self

)

Get the current calibration uncertainty obtained during calibration process.

Returns

sl.GNSS_FUSION_STATUS representing current initialisation status.

Output yaw uncertainty.

Output position uncertainty.

get_geo_tracking_calibration()

Transform get_geo_tracking_calibration

(

self

)

Get the calibration found between VIO and GNSS.

Returns

sl.Transform is the calibration found between VIO and GNSS during calibration process.

enable_spatial_mapping()

FUSION_ERROR_CODE enable_spatial_mapping	(		self,
			SpatialMappingFusionParameters,
			parameters
	)

Initializes and starts the spatial mapping processes.

The spatial mapping will create a geometric representation of the scene based on both tracking data and 3D point clouds.

The resulting output can be a Mesh or a FusedPointCloud. It can be be obtained by calling retrieve_spatial_map_async(). Note that retrieve_spatial_map_async() should be called after request_spatial_map_async().

Parameters

parameters

The structure containing all the specific parameters for the spatial mapping. Default: a balanced parameter preset between geometric fidelity and output file size. For more information, see the SpatialMappingParameters documentation.

Returns

SUCCESS if everything went fine, FUSION_ERROR_CODE::FAILURE otherwise.

Note

The tracking (enable_positional_tracking()) needs to be enabled to use the spatial mapping.

Lower SpatialMappingParameters.range_meter and SpatialMappingParameters.resolution_meter for higher performance.

Warning

This fuction is only available for INTRA_PROCESS communication type.

request_spatial_map_async()

None request_spatial_map_async

(

self

)

Starts the spatial map generation process in a non blocking thread from the spatial mapping process.

The spatial map generation can take a long time depending on the mapping resolution and covered area. This function will trigger the generation of a mesh without blocking the program. You can get info about the current generation using get_spatial_map_request_status_async(), and retrieve the mesh using request_spatial_map_async(...) .

Note

Only one mesh can be generated at a time. If the previous mesh generation is not over, new calls of the function will be ignored.

get_spatial_map_request_status_async()

FUSION_ERROR_CODE get_spatial_map_request_status_async

(

self

)

Returns the spatial map generation status.

This status allows to know if the mesh can be retrieved by calling retrieve_spatial_map_async().

Returns

SUCCESS if the mesh is ready and not yet retrieved, otherwise FUSION_ERROR_CODE::FAILURE.

See request_spatial_map_async() for an example.

retrieve_spatial_map_async()

FUSION_ERROR_CODE retrieve_spatial_map_async	(		self,
			py_mesh
	)

Retrieves the current generated spatial map.

After calling request_spatial_map_async(), this method allows you to retrieve the generated mesh or fused point cloud.
The Mesh or FusedPointCloud will only be available when get_spatial_map_request_status_async() returns FUSION_ERROR_CODE.SUCCESS.

Parameters

py_mesh[out]

: The Mesh or FusedPointCloud to be filled with the generated spatial map.

Returns

FUSION_ERROR_CODE.SUCCESS if the mesh is retrieved, otherwise FUSION_ERROR_CODE.FAILURE.

Note

This method only updates the necessary chunks and adds the new ones in order to improve update speed.

Warning

You should not modify the mesh / fused point cloud between two calls of this method, otherwise it can lead to a corrupted mesh / fused point cloud. See request_spatial_map_async() for an example.

disable_spatial_mapping()

None disable_spatial_mapping

(

self

)

Disables the spatial mapping process.

The spatial mapping is immediately stopped.

If the mapping has been enabled, this function will automatically be called by close().

Note

This function frees the memory allocated for the spatial mapping, consequently, the spatial map cannot be retrieved after this call.