How can I optimize the ZED SDK AI models manually?

The ZED SDK provides AI models for depth estimation (NEURAL, NEURAL LIGHT, and NEURAL PLUS depth modes), Object Detection and Tracking, and Human Body Detection and Tracking (skeleton tracking).

The AI models are not included in the installer of the ZED SDK because of their size, but they can be downloaded from our servers during the installation and optimized for the model of the GPU available on the host device.

If you decide not to download and optimize the AI models during the installation of the ZED SDK, the process is performed automatically by the ZED SDK the first time that the required AI model is used.

There are situations where the AI models must be cleaned and optimized manually:

  • the GPU of the host device has been replaced.
  • the ZED SDK has been updated to a new version.
  • the download process failed for unexpected reasons (e.g. power outage, internet connection drop, etc.).
  • the optimization process stopped for unexpected reasons.
  • unexpected behavior of the ZED SDK processing (e.g. wrong depth map in NEURAL depth modes, wrong object detections, wrong skeleton detection, etc.).

Note: all the operations described below can also be performed from the graphical interface of the ZED Diagnostics tool by clicking on the AI MODELS button, available after completing a diagnostic test. See the ZED Diagnostics documentation for more information. On headless devices (e.g. NVIDIA Jetson without a display), add the -c option to run the tool in command-line mode.

Clean the AI models

Open a shell console.

  • Linux command (the current folder is not relevant):
 
$ ZED_Diagnostic -aic
  • Windows commands:
 
> cd "C:\Program Files (x86)\ZED SDK\tools\"
> ."\ZED Diagnostic.exe" -aic

This removes all the downloaded and optimized AI models. It is useful to free up disk space, or to reset the optimizations if something is not working properly.

Download and optimize the NEURAL depth model

Open a shell console.

  • Linux command (the current folder is not relevant):
 
$ ZED_Diagnostic -nrlo
  • Windows commands:
 
> cd "C:\Program Files (x86)\ZED SDK\tools\"
> ."\ZED Diagnostic.exe" -nrlo

Download and optimize the NEURAL LIGHT depth model

Open a shell console.

  • Linux command (the current folder is not relevant):
 
$ ZED_Diagnostic -nrlo_light
  • Windows commands:
 
> cd "C:\Program Files (x86)\ZED SDK\tools\"
> ."\ZED Diagnostic.exe" -nrlo_light

Download and optimize the NEURAL PLUS depth model

Open a shell console.

  • Linux command (the current folder is not relevant):
 
$ ZED_Diagnostic -nrlo_plus
  • Windows commands:
 
> cd "C:\Program Files (x86)\ZED SDK\tools\"
> ."\ZED Diagnostic.exe" -nrlo_plus

Download and optimize all the AI-based depth models

Open a shell console.

  • Linux command (the current folder is not relevant):
 
$ ZED_Diagnostic -nrlo_all
  • Windows commands:
 
> cd "C:\Program Files (x86)\ZED SDK\tools\"
> ."\ZED Diagnostic.exe" -nrlo_all

Download and optimize a single AI model

Open a shell console.

  • Linux command (the current folder is not relevant):
 
$ ZED_Diagnostic -ais X
  • Windows commands:
 
> cd "C:\Program Files (x86)\ZED SDK\tools\"
> ."\ZED Diagnostic.exe" -ais X

Replace "X" with an integer number according to the sl::AI_MODELS enum:

  • 0 - MULTI_CLASS_DETECTION: related to sl::OBJECT_DETECTION_MODEL::MULTI_CLASS_BOX_FAST
  • 1 - MULTI_CLASS_MEDIUM_DETECTION: related to sl::OBJECT_DETECTION_MODEL::MULTI_CLASS_BOX_MEDIUM
  • 2 - MULTI_CLASS_ACCURATE_DETECTION: related to sl::OBJECT_DETECTION_MODEL::MULTI_CLASS_BOX_ACCURATE
  • 3 - HUMAN_BODY_FAST_DETECTION: related to sl::BODY_TRACKING_MODEL::HUMAN_BODY_FAST
  • 4 - HUMAN_BODY_MEDIUM_DETECTION: related to sl::BODY_TRACKING_MODEL::HUMAN_BODY_MEDIUM
  • 5 - HUMAN_BODY_ACCURATE_DETECTION: related to sl::BODY_TRACKING_MODEL::HUMAN_BODY_ACCURATE
  • 6 - HUMAN_BODY_38_FAST_DETECTION: related to sl::BODY_TRACKING_MODEL::HUMAN_BODY_FAST with sl::BODY_FORMAT::BODY_38
  • 7 - HUMAN_BODY_38_MEDIUM_DETECTION: related to sl::BODY_TRACKING_MODEL::HUMAN_BODY_MEDIUM with sl::BODY_FORMAT::BODY_38
  • 8 - HUMAN_BODY_38_ACCURATE_DETECTION: related to sl::BODY_TRACKING_MODEL::HUMAN_BODY_ACCURATE with sl::BODY_FORMAT::BODY_38
  • 9 - PERSON_HEAD_DETECTION: related to sl::OBJECT_DETECTION_MODEL::PERSON_HEAD_BOX_FAST
  • 10 - PERSON_HEAD_ACCURATE_DETECTION: related to sl::OBJECT_DETECTION_MODEL::PERSON_HEAD_BOX_ACCURATE
  • 11 - REID_ASSOCIATION: related to sl::BatchParameters::enable
  • 12 - NEURAL_LIGHT_DEPTH: related to sl::DEPTH_MODE::NEURAL_LIGHT
  • 13 - NEURAL_DEPTH: related to sl::DEPTH_MODE::NEURAL
  • 14 - NEURAL_PLUS_DEPTH: related to sl::DEPTH_MODE::NEURAL_PLUS

Note: the numeric IDs follow the order of the sl::AI_MODELS enum and can change between major ZED SDK versions (e.g. the depth model IDs changed when the NEURAL LIGHT depth mode was introduced). Always verify the list matching your installed ZED SDK version by running the tool with the -h flag.

Download all the AI models without optimizing them

This is useful to prepare an offline deployment; the optimization will be performed later, directly on the target device.

Open a shell console.

  • Linux command (the current folder is not relevant):
 
$ ZED_Diagnostic -aid
  • Windows commands:
 
> cd "C:\Program Files (x86)\ZED SDK\tools\"
> ."\ZED Diagnostic.exe" -aid

Download and optimize all the AI models

This process can take a while because all the AI models will be downloaded and optimized in a single step.

Open a shell console.

  • Linux command (the current folder is not relevant):
 
$ ZED_Diagnostic -aio
  • Windows commands:
 
> cd "C:\Program Files (x86)\ZED SDK\tools\"
> ."\ZED Diagnostic.exe" -aio

For more information about all the available ZED Diagnostics commands, you can use the -h flag or refer to the ZED Diagnostics documentation.