Why is my application frozen when using the NEURAL depth mode?

Starting with ZED SDK v5.0, all the available depth modes are AI-based and powered by our TERRA AI engine:

 
sl::InitParameters::depth_mode = sl::DEPTH_MODE::NEURAL_LIGHT
sl::InitParameters::depth_mode = sl::DEPTH_MODE::NEURAL        // default
sl::InitParameters::depth_mode = sl::DEPTH_MODE::NEURAL_PLUS

You can find a detailed description of each depth mode in the Depth Modes documentation.

Each depth AI model, like the other AI models available with the Object Detection and Body Tracking modules of the ZED SDK, must be downloaded from the StereoLabs servers and optimized for the GPU installed on the host device the first time it is used. The ZED SDK automatically performs these operations.

The download is required because each AI model takes a significant amount of disk space; including all of them in the ZED SDK installer would result in a very large installer package.

The optimization step allows the AI engine of the ZED SDK to reach full performance by adapting the model to the architecture of the NVIDIA GPU installed on the host machine.

This procedure must be executed only one time after installing a new version of the ZED SDK, for each depth mode that you use.

Downloading and optimizing an AI model can take several minutes, depending on the GPU model and the internet connection speed; if this phase is not correctly handled by your application, it will result in a not responding user interface.

How to avoid the freeze

We suggest downloading and optimizing the depth AI models before starting your application by running the ZED Diagnostic tool from the command line:

  • Windows (from C:\Program Files (x86)\ZED SDK\tools\):
 
"ZED Diagnostic.exe" -nrlo_all
  • Linux:
 
ZED_Diagnostic -nrlo_all

The -nrlo_all option downloads and optimizes all the AI-based depth modes. You can also optimize a single depth mode:

  • -nrlo downloads and optimizes the NEURAL depth model
  • -nrlo_plus downloads and optimizes the NEURAL PLUS depth model
  • -nrlo_light downloads and optimizes the NEURAL LIGHT depth model

If you plan to use other AI modules (Object Detection, Body Tracking), you can download and optimize all the AI models at once with the -aio option.

Alternatively, you can use the GUI of the ZED Diagnostic tool: after running a diagnostic test, click on the AI MODELS button, select the models to optimize, and click Optimize Selection. The AI models optimization can also be triggered by the GUI of the ZED Depth Viewer tool.

For more information about all the available options, see the ZED Diagnostics documentation.

Note: if an optimized model becomes corrupted or the optimization gets stuck, you can reset all the downloaded and optimized AI models with the -aic option (or the CLEAR ALL MODELS button in the GUI), then run the optimization again.

Note: the AI-based depth modes require an NVIDIA GPU with CUDA support. The latest ZED SDK version is available on the Developers page.