Adding Object Detection in ROS Object Detection with RVIZ In this tutorial, you will learn in detail how to configure your own RVIZ session to see only the video data that you require. To visualize the detected objects published by the ZED node, you can use the MarkerArray plugin of Rviz. Marker Array The MarkerArray plugin allows you to visualize a set of markers published by a node. The ZED node, for each detected object, publishes the relative 3D bounding box and label with label_id. Key parameter: Marker topic: Selects the Marker Array topic to be visualized By expanding Namespaces, you can select/deselect what will be visible: bbox: Enable/Disable the object bounding box label: Enable/Disable the object bounding label and lael_id Object list subscribing in C++ In this tutorial, you will learn how to write a simple C++ node that subscribes to messages of type zed_wrapper/objects. This lets you retrieve the list of detected object published by the ZED node for each camera frame. Introduction Use this command to connect the ZED 2 camera to the ROS network: $ roslaunch zed_wrapper zed2.launch The ZED node will start to publish object detection data in the network only if there is another node that subscribes to the relative topic and if the Object Detection module has been started. Note: The Object Detection module in the ZED wrapper can start automatically only if the parameter object_detection/od_enabled in params/zed2.yaml is set to true (default false). To start manually the module manually it is possible to use the service start_object_detection Running the tutorial If you properly followed the ROS Installation Guide, the executable of this tutorial has been compiled and you can run the subscriber node using this command: $ rosrun zed_obj_det_sub_tutorial zed_obj_det_sub If the ZED node is running, and a ZED 2 is connected or you have loaded an SVO file, you will receive the following stream of messages confirming that you have correctly subscribed to the ZED image topics: [ INFO] [1576766230.108543127]: ***** New object list ***** [ INFO] [1576766230.108605109]: Person [5] - Pos. [5.95855,5.95855,0.11876] [m]- Conf. 98.9455 - Tracking state: 1 [ INFO] [1576766230.202101772]: ***** New object list ***** [ INFO] [1576766230.202158329]: Person [5] - Pos. [5.88035,5.88035,0.0847497] [m]- Conf. 99.1507 - Tracking state: 1 [ INFO] [1576766230.202176510]: Person [9] - Pos. [19.4986,19.4986,1.64174] [m]- Conf. 64.1317 - Tracking state: 1 [ INFO] [1576766230.295807403]: ***** New object list ***** [ INFO] [1576766230.295883820]: Person [5] - Pos. [5.79394,5.79394,0.0612211] [m]- Conf. 99.2838 - Tracking state: 1 [ INFO] [1576766230.388406957]: ***** New object list ***** [ INFO] [1576766230.388482749]: Person [5] - Pos. [5.68706,5.68706,0.0395349] [m]- Conf. 99.5577 - Tracking state: 1 [ INFO] [1576766230.388513421]: Person [9] - Pos. [19.5791,19.5791,1.64634] [m]- Conf. 69.3917 - Tracking state: 1 [ INFO] [1576766230.482241321]: ***** New object list ***** [ INFO] [1576766230.482301343]: Person [5] - Pos. [5.57186,5.57186,0.0168457] [m]- Conf. 99.4878 - Tracking state: 1 [ INFO] [1576766230.482320918]: Person [9] - Pos. [19.6178,19.6178,1.65349] [m]- Conf. 56.3891 - Tracking state: 1 [ INFO] [1576766230.574966895]: ***** New object list ***** [ INFO] [1576766230.575027927]: Person [5] - Pos. [5.44794,5.44794,0.00368121] [m]- Conf. 99.7266 - Tracking state: 1 [ INFO] [1576766230.575079817]: Person [9] - Pos. [19.6693,19.6693,1.68823] [m]- Conf. 56.3891 - Tracking state: 2 [ INFO] [1576766230.672211835]: ***** New object list ***** [ INFO] [1576766230.672272723]: Person [5] - Pos. [5.315,5.315,-0.000642871] [m]- Conf. 99.3527 - Tracking state: 1 [ INFO] [1576766230.672292537]: Person [9] - Pos. [19.7051,19.7051,1.71357] [m]- Conf. 56.3891 - Tracking state: 2 [...] Where the “Tracking state” values can be decoded as: 0 -> OFF (object not valid) 1 -> OK 2 -> SEARCHING (occlusion occurred, trajectory is estimated) The code The source code of the subscriber node zed_obj_det_sub_tutorial.cpp: #include <ros/ros.h> #include <zed_wrapper/object_stamped.h> #include <zed_wrapper/objects.h> /** * Subscriber callback */ void objectListCallback(const zed_wrapper::objects::ConstPtr& msg) { ROS_INFO( "***** New object list *****"); for(int i=0; i<msg->objects.size();i++) { if(msg->objects[i].label_id == -1) continue; ROS_INFO_STREAM( msg->objects[i].label << " [" << msg->objects[i].label_id << "] - Pos. [" << msg->objects[i].position.x << "," << msg->objects[i].position.x << "," << msg->objects[i].position.z << "] [m]" << "- Conf. " << msg->objects[i].confidence << "" << " - Tracking state: " << static_cast<int>(msg->objects[i].tracking_state) ); } } /** * Node main function */ int main(int argc, char** argv) { ros::init(argc, argv, "zed_obj_det_sub_tutorial"); ros::NodeHandle n; // Subscriber ros::Subscriber subObjList= n.subscribe("/zed/zed_node/obj_det/objects", 1, objectListCallback); ros::spin(); return 0; } The code explained The following is a brief explanation about the above source code: void objectListCallback(const zed_wrapper::objects::ConstPtr& msg) { ROS_INFO( "***** New object list *****"); for(int i=0; i<msg->objects.size();i++) { if(msg->objects[i].label_id == -1) continue; ROS_INFO_STREAM( msg->objects[i].label << " [" << msg->objects[i].label_id << "] - Pos. [" << msg->objects[i].position.x << "," << msg->objects[i].position.x << "," << msg->objects[i].position.z << "] [m]" << "- Conf. " << msg->objects[i].confidence << "" << " - Tracking state: " << static_cast<int>(msg->objects[i].tracking_state) ); } } This callback is executed when the subscriber node receives a message of type zed_wrapper/objects that matches the subscribed topic. The parameter of the callback is a boost::shared_ptr to the received message. This means you don’t have to worry about memory management. The callback code is very simple and demonstrates how to access the fields in a message; in this case, the object list and for each object its label and label_id, the position and the tracking_state. The main function is very standard and is explained in details in the “Talker/Listener” ROS tutorial. The most important lesson of the above code is how the subscribers are defined: // Subscriber ros::Subscriber subObjList= n.subscribe("/zed/zed_node/obj_det/objects", 1, objectListCallback); A ros::Subscriber is a ROS object that listens on the network and waits for its own topic message to be available. When a message is received, it executes the callback assigned to it. We declared a single subscriber to the /zed/zed_node/obj_det/objects topic that calls the objectListCallback function when it receives a message of type zed_wrapper/objects that matches that topic Conclusion The full source code of this tutorial is available on GitHub in the zed_obj_det_sub_tutorial sub-package. Along with the node source code are the package.xml and CMakeLists.txt files that complete the tutorial package.