Object Detection Module

Classes

class  ObjectData
 Contains data of a detected object such as its bounding_box, label, id and its 3D position. More...
 
class  CustomBoxObjectData
 Container to store the externally detected objects. The objects can be ingested using Camera::ingestcustomBoxObjects() functions to extract 3D information and tracking over time. More...
 
class  Objects
 Contains the result of the object detection module. The detected objects are listed in object_list. More...
 
class  ObjectsBatch
 Contains batched datas of a detected object. More...
 
class  BatchParameters
 sets batch trajectory parameters More...
 
class  ObjectDetectionParameters
 Sets the object detection parameters. More...
 
class  ObjectDetectionRuntimeParameters
 Sets the object detection runtime parameters. More...
 
class  AI_Model_status
 contains AI model status. More...
 

Enumerations

enum class  BODY_PARTS
 semantic of human body parts and order of ObjectData::keypoint for BODY_FORMAT::POSE_18. More...
 
enum class  BODY_PARTS_POSE_34
 semantic of human body parts and order of ObjectData::keypoint for BODY_FORMAT::POSE_34. More...
 
enum class  DETECTION_MODEL
 List available models for detection. More...
 
enum class  AI_MODELS
 Lists available AI moles. More...
 
enum class  OBJECT_CLASS
 Lists available object class. More...
 
enum class  OBJECT_SUBCLASS
 Available object subclass, given as hint, when using object tracking an object can change of OBJECT_SUBCLASS while keeping the same OBJECT_CLASS and id (i.e: frame M: MOTORBIKE, frame N:BICYCLE) More...
 
enum class  OBJECT_TRACKING_STATE
 Lists available object tracking state. More...
 
enum class  OBJECT_ACTION_STATE
 Lists available object action state. More...
 
enum class  OBJECT_FILTERING_MODE
 Lists of supported bounding box preprocessing. More...
 
enum class  BODY_FORMAT
 Lists of supported skeleton body model. More...
 

Functions

static AI_Model_status checkAIModelStatus (AI_MODELS model, int gpu_id=0)
 Check if a corresponding optimized engine is found for the requested Model based on your rig configuration. More...
 
static ERROR_CODE optimizeAIModel (sl::AI_MODELS model, int gpu_id=0)
 Optimize the requested model, possible download if the model is not present on the host. More...
 
String generate_unique_id ()
 Generate a UUID like unique ID to help identify and track AI detections. More...
 
int getIdx (BODY_PARTS part)
 return associated index of each BODY_PART More...
 
int getIdx (BODY_PARTS_POSE_34 part)
 return associated index of each BODY_PARTS_POSE_34 More...
 

Variables

const std::vector< std::pair< BODY_PARTS, BODY_PARTS > > BODY_BONES
 Links of human body keypoints for BODY_FORMAT::POSE_18, useful for display. More...
 
const std::vector< std::pair< BODY_PARTS_POSE_34, BODY_PARTS_POSE_34 > > BODY_BONES_POSE_34
 Links of human body keypoints for BODY_FORMAT::POSE_34, useful for display. More...
 

Enumeration Type Documentation

◆ BODY_PARTS

enum BODY_PARTS
strong

semantic of human body parts and order of ObjectData::keypoint for BODY_FORMAT::POSE_18.

Enumerator
NOSE 
NECK 
RIGHT_SHOULDER 
RIGHT_ELBOW 
RIGHT_WRIST 
LEFT_SHOULDER 
LEFT_ELBOW 
LEFT_WRIST 
RIGHT_HIP 
RIGHT_KNEE 
RIGHT_ANKLE 
LEFT_HIP 
LEFT_KNEE 
LEFT_ANKLE 
RIGHT_EYE 
LEFT_EYE 
RIGHT_EAR 
LEFT_EAR 

◆ BODY_PARTS_POSE_34

enum BODY_PARTS_POSE_34
strong

semantic of human body parts and order of ObjectData::keypoint for BODY_FORMAT::POSE_34.

Enumerator
PELVIS 
NAVAL_SPINE 
CHEST_SPINE 
NECK 
LEFT_CLAVICLE 
LEFT_SHOULDER 
LEFT_ELBOW 
LEFT_WRIST 
LEFT_HAND 
LEFT_HANDTIP 
LEFT_THUMB 
RIGHT_CLAVICLE 
RIGHT_SHOULDER 
RIGHT_ELBOW 
RIGHT_WRIST 
RIGHT_HAND 
RIGHT_HANDTIP 
RIGHT_THUMB 
LEFT_HIP 
LEFT_KNEE 
LEFT_ANKLE 
LEFT_FOOT 
RIGHT_HIP 
RIGHT_KNEE 
RIGHT_ANKLE 
RIGHT_FOOT 
HEAD 
NOSE 
LEFT_EYE 
LEFT_EAR 
RIGHT_EYE 
RIGHT_EAR 
LEFT_HEEL 
RIGHT_HEEL 

◆ DETECTION_MODEL

enum DETECTION_MODEL
strong

List available models for detection.

Enumerator
MULTI_CLASS_BOX 

Any objects, bounding box based

MULTI_CLASS_BOX_ACCURATE 

Any objects, bounding box based, more accurate but slower than the base model

HUMAN_BODY_FAST 

Keypoints based, specific to human skeleton, real time performance even on Jetson or low end GPU cards

HUMAN_BODY_ACCURATE 

Keypoints based, specific to human skeleton, state of the art accuracy, requires powerful GPU

MULTI_CLASS_BOX_MEDIUM 

Any objects, bounding box based, compromise between accuracy and speed

HUMAN_BODY_MEDIUM 

Keypoints based, specific to human skeleton, compromise between accuracy and speed

PERSON_HEAD_BOX 

Bounding Box detector specialized in person heads, particulary well suited for crowded environments, the person localization is also improved

CUSTOM_BOX_OBJECTS 

For external inference, using your own custom model and/or frameworks. This mode disables the internal inference engine, the 2D bounding box detection must be provided

◆ AI_MODELS

enum AI_MODELS
strong

Lists available AI moles.

Enumerator
MULTI_CLASS_DETECTION 
MULTI_CLASS_MEDIUM_DETECTION 
MULTI_CLASS_ACCURATE_DETECTION 
HUMAN_BODY_FAST_DETECTION 
HUMAN_BODY_MEDIUM_DETECTION 
HUMAN_BODY_ACCURATE_DETECTION 
PERSON_HEAD_DETECTION 
REID_ASSOCIATION 
NEURAL_DEPTH 
LAST 

◆ OBJECT_CLASS

enum OBJECT_CLASS
strong

Lists available object class.

Enumerator
PERSON 

For people detection

VEHICLE 

For vehicle detection. It can be cars, trucks, buses, motorcycles etc

BAG 

For bag detection (backpack, handbag, suitcase)

ANIMAL 

For animal detection (cow, sheep, horse, dog, cat, bird, etc)

ELECTRONICS 

For electronic device detection (cellphone, laptop, etc)

FRUIT_VEGETABLE 

For fruit and vegetable detection (banana, apple, orange, carrot, etc)

SPORT 

For sport-related object detection (sportball)

◆ OBJECT_SUBCLASS

enum OBJECT_SUBCLASS
strong

Available object subclass, given as hint, when using object tracking an object can change of OBJECT_SUBCLASS while keeping the same OBJECT_CLASS and id (i.e: frame M: MOTORBIKE, frame N:BICYCLE)

Enumerator
PERSON 

PERSON / PERSON_BODY

PERSON_HEAD 

PERSON

BICYCLE 

VEHICLE

CAR 

VEHICLE

MOTORBIKE 

VEHICLE

BUS 

VEHICLE

TRUCK 

VEHICLE

BOAT 

VEHICLE

BACKPACK 

BAG

HANDBAG 

BAG

SUITCASE 

BAG

BIRD 

ANIMAL

CAT 

ANIMAL

DOG 

ANIMAL

HORSE 

ANIMAL

SHEEP 

ANIMAL

COW 

ANIMAL

CELLPHONE 

ELECTRONICS

LAPTOP 

ELECTRONICS

BANANA 

FRUIT/VEGETABLE

APPLE 

FRUIT/VEGETABLE

ORANGE 

FRUIT/VEGETABLE

CARROT 

FRUIT/VEGETABLE

SPORTSBALL 

SPORT

◆ OBJECT_TRACKING_STATE

enum OBJECT_TRACKING_STATE
strong

Lists available object tracking state.

Enumerator
OFF 

The tracking is not yet initialized, the object ID is not usable

OK 

The object is tracked

SEARCHING 

The object couldn't be detected in the image and is potentially occluded, the trajectory is estimated

TERMINATE 

This is the last searching state of the track, the track will be deleted in the next retreiveObject

◆ OBJECT_ACTION_STATE

enum OBJECT_ACTION_STATE
strong

Lists available object action state.

Enumerator
IDLE 

The object is staying static.

MOVING 

The object is moving.

◆ OBJECT_FILTERING_MODE

enum OBJECT_FILTERING_MODE
strong

Lists of supported bounding box preprocessing.

Enumerator
NONE 

SDK will not apply any preprocessing to the detected objects.

NMS3D 

SDK will remove objects that are in the same 3D position as an already tracked object (independant of class ID)

NMS3D_PER_CLASS 

SDK will remove objects that are in the same 3D position as an already tracked object of the same class ID.

◆ BODY_FORMAT

enum BODY_FORMAT
strong

Lists of supported skeleton body model.

Enumerator
POSE_18 

18 keypoint model of COCO 18.

Note
local keypoint angle and position are not available with this format
POSE_34 

34 keypoint model.

Note
local keypoint angle and position are available
Warning
The SDK will automatically enable fitting.

Function Documentation

◆ checkAIModelStatus()

static AI_Model_status checkAIModelStatus ( AI_MODELS  model,
int  gpu_id = 0 
)
static

Check if a corresponding optimized engine is found for the requested Model based on your rig configuration.

Returns
The status of the given model for the specified GPU

◆ optimizeAIModel()

static ERROR_CODE optimizeAIModel ( sl::AI_MODELS  model,
int  gpu_id = 0 
)
static

Optimize the requested model, possible download if the model is not present on the host.

Returns
SUCCESS if the model is well optimized.

◆ generate_unique_id()

String sl::generate_unique_id ( )

Generate a UUID like unique ID to help identify and track AI detections.

◆ getIdx() [1/2]

int sl::getIdx ( BODY_PARTS  part)
inline

return associated index of each BODY_PART

◆ getIdx() [2/2]

int sl::getIdx ( BODY_PARTS_POSE_34  part)
inline

return associated index of each BODY_PARTS_POSE_34

Variable Documentation

◆ BODY_BONES

const std::vector<std::pair<BODY_PARTS, BODY_PARTS> > BODY_BONES

Links of human body keypoints for BODY_FORMAT::POSE_18, useful for display.

◆ BODY_BONES_POSE_34

const std::vector<std::pair<BODY_PARTS_POSE_34, BODY_PARTS_POSE_34> > BODY_BONES_POSE_34

Links of human body keypoints for BODY_FORMAT::POSE_34, useful for display.