STEREOLABS
Store

How to Make Mixed Reality VR Videos in Unity

Part 7: Virtual Lighting 

Now we’re going to make it so that the virtual lights in your game affect your real-world subjects, as well. That means bright flashes, colors and shadows on your real-world player from lights that don’t actually exist. This adds another layer of realism to the scene, as seen above, and will put your final product a cut above most mixed reality productions.

Gif of mixed reality scene with and without virtual lighting
The left scene has virtual lighting applied to the real world. The right scene does not.

Doing this is straightforward – we’ll simply add a script to each light we want to affect the real world, and the Plugin will handle the rest. But there are some limitations you must keep in mind:

  • You can have a maximum of 8 lights set to work on the real world at the same time
  • Only Directional lights can cast shadows – we recommend exactly one per scene to get the desired effect
  • This only works on Point, Spot and Directional lights. Area lights, emissive materials and lights instantiated by a particle system aren’t supported.

Note: The specifics of lighting a Unity scene is outside the scope of this tutorial, but Unity provides detailed documentation on this topic here. If you are working with a new scene, we suggest experimenting with a Point light, and having at least one Directional light in the scene for shadows.

Select each light you want to shine on the real world in the Hierarchy. Then, in the Inspector, hit Add Component. Then type “ZEDLight” and click on the script when it appears in the list.

Adding ZED Light component

After you’ve done this with all the lights you want, test the scene and see how they look on your subject. You can move the light around in the Scene window at runtime to check its effect at different distances. You can also modify the parameters of the light in the Inspector, such as Color, Intensity and Range.

Matching Virtual and Real Lights

Depending on your real-world subject and lighting, you may find that a light’s effect on virtual objects and real objects differs significantly – such as a torch’s flame brightly illuminating the virtual ground but only slightly illuminating the person holding it.

To fix that, we’ll stop that light from casting on real objects, and instead create a duplicate light that only casts onto real objects, so that we can tweak it without affecting the real world.

First, select the light in question. In the inspector, click on the tiny gear icon at the top right of the ZED Light script. Then click “Remove Component.”

Removing ZED Light component

Then, right click on the light in the Hierarchy and click “Duplicate.” You’ll see a copy of that light appear underneath it. To make sure the light’s stay together, click and drag the new light onto the old one to make it a child of it. That way, if the original light moves, the new one will move as well.

Unity scene with two lights

With the new light selected, rename the new light to something that indicates its specific role, like “Point Light – ZED Clone,” and add the ZEDLight script to it as you did before. Then go to the light’s Culling Mask value and click “Nothing.” This will tell it not to affect real objects.

Now run the scene, and with the new light selected, tweak its settings in the Inspector. Most likely, you’ll be tweaking Range and Intensity. Once the lighting on the real and virtual objects are similar enough, write down the values you changed and stop the scene. Re-add those values, and you’re finished.

Next: Part 8: Final Output

Tags