The ZED is a passive stereo vision camera that reproduces the way human binocular vision works. It embeds two synchronized image sensors separated by a fixed, factory-calibrated distance called the baseline. By matching the same visual features in the left and right images and measuring their horizontal shift (the disparity), the ZED computes the distance of every pixel through triangulation, building a dense three-dimensional model of the observed scene in real time.
Because the ZED relies on ambient light and natural scene texture instead of projecting infrared patterns or laser pulses, it is a fully passive sensor; multiple cameras can operate side by side without interfering with each other, indoors and outdoors, even in direct sunlight.
How is the depth map generated?
The camera outputs a high-resolution, hardware-synchronized side-by-side color video stream (left and right views). The ZED SDK running on the host machine rectifies the two images using the factory calibration, performs stereo matching on the GPU to compute a disparity map, and converts it into a metric depth map. Modern SDK versions also offer AI-based stereo matching (NEURAL depth modes) that significantly improves robustness on low-texture and reflective surfaces.
Developers have simultaneous access to:
- The synchronized left and right color images
- The dense depth map and confidence map
- A 3D point cloud with color and surface normals
All outputs are spatially and temporally aligned; there is no need to register a separate RGB sensor to the depth data, as required by most active depth cameras.
How does stereo vision compare to other depth sensing technologies?
Active depth sensors based on structured light or Time-of-Flight are typically limited to short range and indoor use, since sunlight overwhelms their infrared signal. LiDAR performs well outdoors but provides sparse geometric data without color and at a higher cost.
Passive stereo combines the strengths of both worlds: long-range depth perception (up to 20 m and beyond, depending on the model and baseline), full-resolution color and depth from the same sensor pair, and reliable operation both indoors and outdoors. This makes ZED cameras suitable for robotics, AR/VR, video analytics, security, VFX, logistics, agriculture, and 3D scanning applications.
Can I use the ZED for people counting and tracking?
Yes. The long depth range of stereo vision makes the ZED a good fit for people counting and tracking in large areas. In addition, the ZED SDK includes dedicated Object Detection and Body Tracking modules that combine AI detection with 3D depth information to localize and track people in metric space, with unique IDs and 3D skeleton data. Keep in mind that fine segmentation of small body parts, such as fingers, is limited at long range because disparity resolution decreases with distance.
Best practices for stereo depth sensing
- Working range: each model has an optimal range defined by its baseline and focal length. As a rule of thumb, depth accuracy is highest in the near range; place the camera as close to the region of interest as the application allows.
- Scene texture: stereo matching needs visual features. Homogeneous, textureless, or specular surfaces (white walls, green screens, glass, mirrors) can generate unstable or missing depth. Adding texture or using the NEURAL depth mode mitigates this.
- Lighting: the ZED uses color images, not IR, for depth perception; avoid very low-light environments, or add scene illumination. Outdoors, direct sunlight is not a problem.
- Fast motion: to capture fast movements, use the high frame rate modes (for example HD720 @ 60 FPS or VGA/SVGA @ 100 FPS on USB models) to reduce motion blur, which degrades stereo matching.
- Cable extension: the USB 3.0 connection can be extended up to 15 m with active USB 3.0 extension cables, and up to 100 m using optical-fiber USB 3.0 extenders. For industrial-grade long cabling and multi-camera setups, the ZED X family uses a GMSL2 interface designed for embedded systems.
What is the depth accuracy?
Stereo depth is estimated by triangulation from the disparity image. The depth resolution follows this relation:
Dr = Z² * alpha
where Z is the distance to the target and alpha is a constant that depends on the baseline, the focal length, and the matching sub-pixel precision.
In practice, depth error grows quadratically with distance: it is typically around 1% of the distance in the near range and can reach several percent at the far end of the range. Accuracy can also be affected by outlier measurements on homogeneous and textureless surfaces, which usually produce temporal instability in the depth values; the SDK confidence map can be used to filter these unreliable pixels.