Getting Started with ZED Live Link

The ZED Live Link allows developers to send camera tracking informations as well as skeleton data into Unreal Engine 4 using the Live Link Plugin integrated into UE4. It also contains a Unreal Engine sample project already set-up to receive this data. This way, you can control a Skeletal Mesh in real-time using the Skeleton tracking module of the ZED SDK.

You don’t need to use the ZED Plugin for Unreal, only the ZED Live Link plugin and the Unreal project we provide you. You can have an overview of the solution on the diagram below:

The machine running ZED Live Link will use the ZED SDK, so it should be installed, download it here. The Unreal Project can run on a separate machine (Windows or Linux).

Live Link is a plugin designed to stream animation data coming from external sources to UE4. You can get more information about this plugin in the UE4 Live Link doc. Our custom ZED Live Link plugin is therefore designed to stream ZED SDK skeleton data to UE4.

UE4 project

We provide you an Unreal project that already contains the different assets necessary to animate avatars properly using the ZED Live Link skeleton data. It requires Unreal Engine 4.26. Some explanations about this project are available here: How Our Project Works.

You will be able to use directly this project by hitting play while the ZED Live Link Plugin is running and see a default avatar animate. We are showing how to add your own models here: Animate New Avatar.

In this tutorial, you’ll learn how to use our ZED Live Link Plugin, connect it to our Unreal Project and animate a default avatar with the Live Link skeleton data. You must follow these steps:

1. Open the Unreal Project

Open our ZEDUnrealLiveLink.uproject project. You’ll see the following empty scene:

In the Content folder of the editor, you’ll see that there are already 3 Animation Blueprints, that correspond to 3 different avatars already imported in the project, as well as 2 assets called Avatar_livelink and MixamoRemap. We’ll see in the page Animate New Avatar what’s their use and how you can add your own assets to animate new avatars.

This project already handles the Live Link skeleton data in order to instantiate correctly avatars and place them in the scene with the corresponding joint positions and orientations. If you want to see how this is done, you can check the project Level Blueprint. The section How our project works explains more in detail the different assets and the transformations implemented in our project to use the Live Link skeleton data.

You can follow these steps if the Live Link Plugin is not enabled yet in your project.

  1. Inside your project, In the Menu Bar under Edit, select Plugins.

  1. Under the Animation section, click Enabled for Live Link, and Yes on the confirmation window, then restart the Editor.

At this point, you must run the ZED Live Link plugin executable. It will call the ZED SDK in order to compute skeleton data and stream it to the Unreal project. So you must make sure to have the ZED SDK installed on the machine that will run the ZED Live Link plugin.

To know how to build the ZED Live link plugin, take a look at our Build Zed Live Link documentation page.

  • Open a terminal in the folder containing the ZED Live Link executable, and run
$ ./ZEDLiveLink

To send data from a specific camera, add its serial number as arguments. For example, to connect the camera whose serial number is 10028418, run :

$ ./ZEDLiveLink 10028418

You can see the connection status in the terminal.

  • On Windows :

  • On Linux :

This step is normally handled automatically by the Unreal Project provided. If you have data coming from several Live Link sources, you might need to do the following to make sure you’re connected to the right source.

  1. Inside the UE4 Editor, from the Menu Bar under Window, select Live Link.

  1. In the Live Link window, click the Source button, then under Message Bus Source, select your ZED source.

5. Hit Play in Unreal

With the project you’re provided, once the ZED Live Link plugin is running, you can animate a default avatar by directly hitting play in Unreal. You will see avatars animated with skeleton data coming from the connected Live Link source:

Troubleshooting

  • Your firewall might block the data stream. If you do not see the ZED Source in the Live Link window, try to disable it.

  • If the ZED Source is not yet detected in UnrealEngine, enable Enable by default in Edit -> Project Settings -> UDP Messaging -> Enable by default

  • On Linux, if the ZED Live Link plugin executable crashes at the start, try to run the ldd command onto the sl_zed_c.so library :
$ ldd sl_zed_c.so

It will show all the dependencies required by the .so and allow you to install anything that might be missing (for example lib-usb).

Note that the c wrapper used for the Live link plugin is also available here : https://github.com/stereolabs/zed-c-api.

If you encounter issues running the live link plugin, do not hesitate to build the wrapper yourself and place it in the lib/win64 or /linux folder.