> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.stereolabs.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.stereolabs.com/_mcp/server. # Environmental Mapping with Unity In this tutorial, you will learn how to build a procedural mesh of your room and create applications with real world navigation and physics. ## Introduction [Spatial Mapping](/docs/development/zed-sdk/modules/spatial-mapping/) captures a mesh of your environment, allowing physical interactions between the real and virtual world. You can use Unity's [Runtime Navigation System](https://learn.unity.com/tutorial/navmesh-baking) to bake the mesh into a **navigation mesh** (or "NavMesh" for short). This structure describes the walkable surfaces in your space and tells characters where they can walk/move in the real world. Spatial mapping also allows you to use Unity's built-in **physics** engine, adding convincing physical behaviour to virtual objects. They can collide with the real world, bounce off of it and be affected by gravity and other forces present in the real world. ## Create a new Project * Select **File** > **New Project** and click **Create project**. * Make sure the ZED SDK is installed, and [install the ZED Unity plugin](/docs/integrations/unity/#step-3-import-the-plugin) through the Package Manager. * Select **File > New Scene**. * In the Hierarchy panel, delete the Main Camera. * Drag the **ZED\_Rig\_Mono** prefab (**Packages** -> **ZED SDK** -> **Prefabs**) into the Hierarchy. ## Mapping an Area ZEDManager's Inspector provides a UI for configuring, starting, and stopping spatial mapping. Once started, it will create a procedural mesh within Unity that can be used at runtime, or saved for later. * Select the **ZED\_Rig\_Mono** prefab you just dropped into the Hierarchy. * In the Inspector, Click the triangle next to **Spatial Mapping** to reveal the options. ![image](/_fern-img/0880f516617a9fb637d09695013936c6871f8b52671a56ce57cabfd0b4377fd9.webp) For more details on Spatial Mapping parameters, read this [section](/docs/development/zed-sdk/modules/spatial-mapping/#spatial-mapping-parameters). We recommend using the following set of parameters: * Resolution = `MEDIUM` * Range = `MEDIUM` * Filtering = `LOW` Now, let's start mapping the area. * Launch the scene. * Press the **Start Spatial Mapping** button to launch the mapping. Move the ZED around to capture a mesh of your environment - a blue wireframe will indicate which areas you have scanned. * When finished, click on **Stop Spatial Mapping** to stop updating the mesh and start mesh filtering. ### Mesh Holder In the Hierarchy, you will now see a **ZED Mesh Holder** object storing the mesh of the real-world environment. If you didn't select the *Save Mesh* option, the mesh will be temporarily stored in RAM and deleted at the end of the session. Expand the mesh holder to see the different *chunks* (submeshes) that compose the global mesh. During procedural meshing, the surrounding area is split up into regions. Each region corresponds to a chunk. When spatial mapping is stopped, these chunks are merged to create larger submeshes. The submeshes have a maximum vertex count of 65535, which is the maximum number of vertices of a Unity mesh using the default 16-bit index format. The chunks are used to generate the NavMesh and handle complex collisions with the real world. ## Using the Navigation Mesh To add a character that navigates the real world, we will use Unity's navigation system. Unity can create a Navigation Mesh which is a simplified plane that describes the walkable surfaces of a game. This allows a character to automatically find its path through obstacles. As of version 5.6, Unity supports Runtime NavMesh building, which allows our plugin to automatically bake NavMesh at runtime using the environment geometry. ### Building the NavMesh The Spatial Mapping sample includes a `NavMeshSurface.cs` script, which builds a **NavMesh Surface** at runtime. A NavMesh Surface represents the walkable area for one [Agent](https://docs.unity3d.com/2023.1/Documentation/Manual/class-NavMeshAgent.html) type. Multiple NavMesh surfaces can be present in a scene. Once mapping is done, the NavMeshSurface component will collect all chunk sources and build a NavMesh for a given Agent Type. * Create a new empty GameObject in the Hierarchy and name it "Navigation". * In the Inspector, click "Add Component" and add a **Nav Mesh Surface** component. * Drag and drop your **ZED\_Rig\_Mono** object from the Hierarchy to the NavMeshSurface's **ZED Rig** field. * Select your *Agent Type*. * Start the scene, and launch Spatial Mapping and map the area. * Press **Stop Spatial Mapping**. The script you just added will generate a NavMesh from the procedural mesh. * Toggle between **Display Mesh** and **Hide Mesh** to show/hide the NavMesh. ![image](/_fern-img/5cc7f4a6434e4277cfc8cdbfcc6fe51e5295321f5acbf373ba2c0205d1891c45.webp) > **Note** > > It is not possible to bake a NavMesh at runtime using *Window > Navigation > Bake* as one would with imported 3D meshes. You need to bake the NavMesh using the scripts provided or Unity's [Components](https://github.com/Unity-Technologies/NavMeshComponents) for navigation. ### Adding Agents Let's place a Bunny character in the scene and make it move in random patterns on the walkable surface. A NavMeshSurface is required for each type of agent, and an agent of one type cannot use surfaces of other types. This is because the navmesh defines the agent's mobility around the scene, and different agents have different mobility: taller enemies can step over bigger objects, flying enemies can ignore terrain, etc. The NavMeshSurface component has a drop-down where you select which agent type to bake for. * Select *Humanoid* Agent Type. * Create an empty GameObject and attach the **EnemyManager** component. The script makes the Bunny move in random patterns on the walkable surface determined by the NavMesh. * Drag the **ZomBunny** prefab (*not* the .fbx file) located in *Assets > Samples > ZED SDK > version > Spatial Mapping > ZomShooter > Prefabs* to the **Enemy Prefab** value in EnemyManager. * Play the scene, start mapping a large area and when you press *Stop Spatial Mapping*, the NavMesh is generated and the Bunny character appears in the scene. The ZomBunny prefab you dragged over is set up with Unity's [NavMeshAgent](https://docs.unity3d.com/ScriptReference/AI.NavMeshAgent.html) script. This allows a Unity GameObject to move around a NavMesh while constrained as expected - it can only walk on flat surfaces that are wide enough for it, can walk up ramps and stairs provided they are not too steep, etc. However, the traditional way of using NavMeshes is to create them before you run a scene. Since we're building the NavMesh at runtime, the *EnemyManager* script will disable the ZomBunny at the start and re-enable it once there's a valid NavMesh present. The ZomBunny prefab also has a custom script called *NavMeshAgentController* which moves it to a random location on the NavMesh once activated. ### Customizing Agents You may notice that the NavMesh is smaller than expected. This is because the default "Humanoid" agent in Unity is built for a larger creature than our bunny. We can tweak that in Unity's Navigation window. * Click on *Window* -> *AI* -> *Navigation* * In the new Navigation tab, click on *Agents* at the top. * Change the *Radius* value from 0.5 to 0.25. ![image](/_fern-img/a74d4f3796ddd2bcd21c18d0cd9bef5e977b39084496eebbbb08f3b606be8dfb.webp) > **Note** > > Depending on your environment, you may also wish to adjust the slope and maximum step height to allow the bunny to walk along more surfaces. Complete documentation on Unity agent parameters is available [here](https://docs.unity3d.com/ScriptReference/AI.NavMeshAgent.html). ## Using Physics At the end of spatial mapping, the mesh is automatically assigned a **MeshCollider** component to resolve collisions. We can now throw virtual balls in the real world and let Unity handle physics and collisions with the underlying mesh. * Add the script **BallLauncher** located in *Assets > Samples > ZED SDK > version > Spatial Mapping > Scripts* to the **Camera\_Left** GameObject. * Select a ball type with the prefab located in *Assets > Samples > ZED SDK > version > Spatial Mapping > Resources > Physics > Sphere.prefab* * Set Message Instruction to any GameObject in the scene - **Camera\_Left** for example. * Play the scene and press the space bar to throw bouncing balls into the real world environment!