Spatial Mapping Best Practices

The Spatial Mapping module of the ZED SDK, also available through the standalone ZEDfu application, enables the 3D scanning of a scene or an object. A colored 3D model (Mesh or Fused Point Cloud) is created by integrating image, depth, and positional tracking data over time, from multiple viewpoints.

For a complete description of the module, see the Spatial Mapping documentation; for the ZEDfu tool, see the ZEDfu documentation.

Supported cameras

Spatial Mapping requires a stereo camera equipped with an inertial sensor (IMU), such as the ZED 2/2i, ZED Mini, ZED X, ZED X Mini, and ZED X Nano.

The following cameras are not supported:

  • the original ZED (out of production), which has no built-in IMU;
  • the ZED X One GS/4K/S cameras, which are monocular. Note that a Dual ZED X One Virtual Stereo Camera setup is supported; in ZEDfu, use the Local Streaming input to receive its data.

Best practices while scanning

  • Move slowly and smoothly. Fast motion introduces motion blur; blurry images lower the quality of both the positional tracking and the reconstructed 3D geometry.
  • Make sure the scene is well lit. Low-light areas increase depth noise and can introduce artifacts and incorrect surfaces in the model.
  • Prefer textured scenes. Untextured, uniform surfaces (bare walls, glass, and so on) provide little visual information for stereo matching and tracking. Thanks to the NEURAL depth modes, indoor scanning now performs much better than with older SDK versions, but rich texture still improves results.
  • Avoid reflective and transparent objects. Mirrors, glass, and shiny surfaces produce invalid depth measurements.
  • Keep an appropriate distance from surfaces. Do not get closer than the minimum depth range of your camera model, and remember that depth accuracy decreases with distance; the mapping range is limited to 20 m for this reason.
  • Close the loop. When scanning a room or an object, try to revisit previously scanned areas; this helps the positional tracking refine the map. Enabling Area Memory allows the SDK to recognize previously mapped areas and improve tracking consistency; see the VSLAM Mapping tutorial.
  • Watch the tracking status. In ZEDfu, if the Positional Tracking status switches to SEARCHING, move the camera back toward an already-mapped area to help the SDK relocalize and continue the map generation.

Choosing the right settings

  • Depth mode: in ZEDfu you can choose between NEURAL LIGHT (best live performance), NEURAL (good compromise between quality and performance), and NEURAL PLUS (highest quality, recommended for OFFLINE processing of SVO files).
  • Map type: choose MESH to generate a set of watertight triangles that can be filtered and textured after scanning, or POINT CLOUD to generate a colored fused point cloud. The point cloud format takes more memory due to the color information attached to each point.
  • Mapping resolution: the resolution can be set between 1 cm and 12 cm. Higher resolutions produce more detailed maps, but require more memory and processing power and slow down map updates; use the lowest resolution that fits your application. The API provides HIGH (2 cm, small areas), MEDIUM (5 cm, balanced), and LOW (8 cm, large areas or collision meshes) presets.
  • Mapping range: the range of the depth data integrated into the map can be set between 2 m and 20 m. Increasing the range allows capturing large volumes quickly, at the cost of accuracy; reducing it improves performance. The API provides NEAR (3.5 m), MEDIUM (5 m), and FAR (10 m) presets.
  • Mesh filtering: after capture, filtering reduces the polygon count while preserving the geometric features. LOW fills holes and cleans outliers; MEDIUM and HIGH also perform mesh decimation. Try the different presets to find the one that best suits your application.
  • Mesh texturing: enable the texture option before scanning to record a subset of the left camera images and project them onto the mesh surface. This improves the visual quality of the model, but requires more processing power and memory. Texturing is available for meshes only.

Live scanning or offline processing

The Spatial Mapping module, including ZEDfu, accepts three types of input:

  • Live Camera: data captured in real time by a connected ZED stereo camera;
  • Local Streaming: data received from a remote camera through a ZED Local Streaming server;
  • SVO File: data recorded with the camera and processed later. Learn more about recording and offline processing.

When processing an SVO file in ZEDfu, we recommend enabling the OFFLINE mode: the full SVO file is preprocessed before the map generation starts, producing a higher-quality map at the cost of longer processing time. Combined with the NEURAL PLUS depth mode, this is the workflow that provides the best possible 3D reconstruction quality.

Note: ZEDfu saves the generated maps in the ZED/Meshes folder in the user's home directory; the full path of the generated map is displayed in the status bar when the generation is complete.

Using the Spatial Mapping API in your application

Everything available in ZEDfu can be integrated into your own application through the ZED SDK. The typical workflow is:

  1. open the camera and enable positional tracking with enablePositionalTracking();
  2. enable the module with enableSpatialMapping(), configuring SpatialMappingParameters (resolution, range, map type, texture);
  3. call grab() in your loop; the module ingests images, depth, and poses in the background;
  4. retrieve the map once with extractWholeSpatialMap(), or continuously with the asynchronous requestSpatialMapAsync() / getSpatialMapRequestStatusAsync() / retrieveSpatialMapAsync() functions;
  5. optionally refine the mesh with Mesh::filter() and Mesh::applyTexture(), then save it with save("mesh.obj");
  6. monitor the module with getSpatialMappingState(): FPS_TOO_LOW indicates that the system cannot reach the framerate required for consistent mapping, and NOT_ENOUGH_MEMORY indicates that the memory limit has been reached; in both cases the module stops integrating new data, but the 3D model can still be extracted.

Requesting and retrieving a spatial map is resource-intensive; to improve performance, do not request a new spatial map too frequently.

For more details, see: