How to Make Mixed Reality VR Videos in Unity
Part 5: Controller Delay
With everything calibrated, the real and virtual controllers will line up. But if you move the controllers around quickly, you may notice the virtual controllers lag behind the real ones slightly.
Above: When he swings quickly, the lack of synchronization makes the lightsaber seemingly float in the air – and not in the cool Luke Skywalker way.
This is because the Constellation/Lighthouse tracking systems of the headsets are designed to update instantaneously, but the ZED has several complicated steps to process each frame, resulting in a delay that’s noticeable when the difference between frames is large (such as when the controllers are moving quickly).
Our plugin has scripts that deal with that. If you’re using Vive/SteamVR, the script has already been added to the ZED_GreenScreen prefab. It’s called ZEDSteamVRControllerManager, and you can see it in the Inspector.
If you’re using Oculus, remove that script by clicking on the gear icon in the top right and clicking “Remove Component. Then, at the bottom, click Add Component. Search for “ZEDOculusControllerManager” and click on it to add it.
Also if using Oculus, you’ll also have to add a custom precompiler directive to your project, which is easier than it sounds. Go to Edit -> Project Settings -> Player. In the Inspector, open Other Settings and under Scripting Define Symbols, add “ZED_OCULUS.” This tells Unity that it’s safe to use some code from our plugin that depends on having the Oculus plugin installed.
If your project has the controllers modeled to be something other than the default controllers, you’ll have to drag the gameobject with those controller models into the “Model To Replace” field. This is because the system works by creating clone objects that trail behind your controllers while your real controllers (which are forced to update by Oculus/SteamVR) get hidden from the ZED. If you do this, the plugin will copy the object you give it instead.
Run the scene. If you’re using Oculus, you’ll note the improvement in delay now that you’ve got the script added. Either way, slide the Latency Compensation slider around to adjust the delay, and test it until the controllers stay aligned even when you move them quickly.
Tip: If your in-game controllers are using models that replicate the real-life controllers exactly, it can be hard to catch if you add too much latency accidentally. You’ll see one controller lagging behind the other, but since you’re moving quickly, you may not be able to tell that it’s the virtual controller that’s lagging behind. Keep this in mind, or perhaps mark your real life controller with colorful tape or something similar to keep them visually distinct.
Note: These scripts use a system of clones to trail your controllers, and forces your controllers into a special layer to hide them from the ZED. This should be fine for most cases. But if your project relies on certain interactions with your controller’s gameobjects, particularly if layers are involved, you may not be able to use these scripts or will have to implement workarounds.
Once you can move the controllers very quickly and have them keep up pace, you’re ready to move on.
