Build your first AR/MR App with Unreal In this tutorial, you will learn how to create a simple mixed reality application. Creating Your Level We’ll start our level from scratch. Delete all the objects in the level - you can do this by selecting everything in the World Outliner under your level name (“Untitled” by default) and hitting Delete. Note: You choose your level’s name the first time you hit Control-S to save. This will also make two files appear in your Content Browser, as seen above. Adding a Light Directional Light We need to add a Directional Light to the scene so that the ZED image is not black. The directional light will not illuminate the real scene, but it will be used to illuminate virtual objects and cast shadows in the real world. Add a Directional Light by going to Lights in the Modes tab and clicking and dragging a Directional Light into the viewport. With the new light selected, go to the Details panel, click on the Eye icon and enable Show All Advanced Details. This will add additional properties to the list. Scroll down until you see Lighting Channels, expand it, and enable Channel 3. This is the lighting channel that affects the real-world image captured by the ZED. Any virtual light with the Lighting Channel 3 enabled will affect the image. Scroll to the top of the Details panel. In the Transform section, set the Rotation values to X = 0, Y = -90, Z = 0. This will orient the directional light straight down, which replicates the orientation of natural light and makes virtual objects look realistic in the real world. Note: If you have a bright, dominant light in your environment from a direction other than up, you can experiment with rotating the light to match it. But remember that it’ll only look right in this environment, and only if you match up the ZED’s orientation with it. Ambient Light Now we need to add some ambient light to the scene. If we don’t, any part of an object that the directional light can’t cast on (such as its underside) will be completely black. In the Modes menu under Lights, drag a Sky Light into the scene. With the new light selected, go to the Details panel. Under Light, change the Source Type to SLS Specified Cubemap. Then under Cubemap, click the drop-down menu. With it open, click on View Options and make sure that Show Plugin Content is enabled. Doing so added a texture from the ZED plugin to the dropdown menu: T_ZED_Cubemap, as seen above. Click that. It’s a neutral cubemap that will tell the light to emit white light from all directions. Next, make the following changes: Set Cubemap Resolution to 64 Set Intensity to 0.2 Disable Cast Shadows Disable Lower Hemisphere Is Solid Setting Up the Scene Now we’ve got to add a Blueprint from the ZED plugin. But by default, Content from plugins is hidden. To fix this, click on View Options at the bottom right of the Content Browser and enable Show Plugin Content. Now click on the folder icon beside Content and click on Stereolabs Content to switch to the plugin’s content folder. In the Content Browser, go to ZED -> Blueprints and drag a BP_ZED_Initializer into the scene. This is the object that sets up your camera and handles communication with your app. As your ZED tracks its position and rotation in your environment, we want it to remember features as it goes to help keep itself stabilized and recover from drift. To do this, go to the Details panel (with the new BP_ZED_Initializer selected), expand Tracking Parameters, and enable Spatial Memory. Adding a Virtual Object We’ve got the lights and the camera. Time to add the action - a virtual object to add to the scene. In the Modes tab under Basic, drag a Sphere into your scene. Once you do, go to the Details tab and set its Location to X = 150, Y = 0, Z = 0. We move it here because the ZED will start at 0,0,0, and we want the sphere to appear in front of the camera. Also set the Scale to X = 0.5, Y = 0.5, Z = 0.5. This will shrink it down to a manageable size to experiment with. Now it’s time to run your scene! You can hit Play at the top to view it from the editor, or click the arrow next to Play at the top and select VR Preview to view it in your VR headset. Once loaded, you’ll see your real environment, but with a virtual sphere floating in it, and a virtual shadow underneath. If in VR mode, put on your headset and see into the reality you’ve augmented with both eyes. Walk around the sphere. Put your hand over and under it. Look away and come back, and it’ll still be there. Move the sphere in the Scene window and watch it fly across your room. You’ve built your first ZED Mini AR app! Tips To keep the ZED Mini’s cable in place and avoid tripping over it, use cable ties to run the ZED Mini’s cable along your headset’s. You should also attach the cable to the headset itself to minimize strain. To see a similar scene already built, download our samples project from our Github page and check out the L_Passthrough level. There’s also plenty of other scenes for getting started that will show you the basics of lighting, spatial mapping, object placement and more.