⚡ ZipDepth ⚡

Bringing Lightweight Zero-Shot Monocular Depth
Anywhere, on Any Device

🏛️ ECCV 2026

Fabio Tosi · Luca Bartolomei · Matteo Poggi · Stefano Mattoccia

University of Bologna

     

---

📢 News

• [Jul 2026] — 🚀 Code and pretrained model released.
• [Jun 2026] — 🎉 ZipDepth accepted at ECCV 2026.

📱 See ZipDepth running live on an iPhone on the project page.

---

ZipDepth is a lightweight zero-shot monocular depth estimation model that achieves the best accuracy–efficiency trade-off among lightweight methods, approaching transformer-based foundation models at a fraction of their cost. It combines reparameterizable convolutions (RepVGG), efficient global attention with learnable tokens, and a compact FPN decoder — all designed to run fast on any device, from edge hardware to server GPUs.

---

🖼️ Qualitative Results

ZipDepth generalizes across diverse domains without any fine-tuning — nighttime driving, outdoor objects, close-up textures, and synthetic content.

RGB	Depth	RGB	Depth

---

📊 Quantitative Results

ZipDepth achieves state-of-the-art accuracy among lightweight embedded models on NYUv2, KITTI, ETH3D, ScanNet, and DIODE, while being significantly more efficient than large pretrained models.

---

🏗️ Architecture

The encoder is organized in four hierarchical stages. Stages 1–2 use RepVGG reparameterizable blocks (3×3 + 1×1 + identity branches fused into a single 3×3 at inference) augmented with Strip Pooling Attention for horizontal/vertical context. Stage 3 adds Squeeze-and-Excitation channel attention and a Global Context Block. Stage 4 deepens the representation with additional RepVGG blocks.

The neck combines SPPF multi-scale pooling with a Cross-Scale Fusion module. The decoder is a lightweight FPN with a Convex Upsampling head for sub-pixel-accurate depth maps.

---

🔧 Installation

Requirements: Python ≥ 3.9, PyTorch ≥ 2.4, CUDA (recommended)

git clone https://github.com/fabiotosi92/ZipDepth
cd ZipDepth

# Create and activate a virtual environment (recommended)
python -m venv venv
source venv/bin/activate

pip install -r requirements.txt
pip install -e .

PyTorch / CUDA: requirements.txt installs the default PyTorch wheel, which on Linux ships with a bundled CUDA runtime. If you need a specific CUDA version (or a CPU-only build), install PyTorch first from the official selector, then run pip install -r requirements.txt.

Optional dependencies

# Faster JPEG decoding (recommended)
pip install PyTurboJPEG

# FLOPs measurement in benchmark
pip install fvcore thop

# ONNX graph simplification during export
pip install onnx onnxsim

---

🗂️ Checkpoints

Pretrained checkpoints are included in the repository under checkpoints/:

File	Upsampling	Params (fused)	Recommended for
zipdepth_base.pth	Convex (unfold)	~6.1 M	GPU / server
zipdepth_base_npu.pth	Convex (unfold-free)	~6.1 M	NPU / mobile / CPU

Both variants share identical encoder and decoder weights. The only difference is the final upsampling step:

• zipdepth_base.pth — uses torch.nn.Unfold for sub-pixel convex upsampling. Best accuracy on GPU.
• zipdepth_base_npu.pth — replaces the unfold operation with an NPU-friendly equivalent. Use this for ONNX export targeting mobile or edge devices.

---

🚀 Quick Start

Single image

python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/im0.jpg \
  --input-size 384

Folder of images

python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/imgs/ \
  --output output/depth/

Video

python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/clip.mp4

---

⚡ Inference

python scripts/infer.py --checkpoint <ckpt> --input <path> [options]

Argument	Default	Description
--checkpoint	required	Path to .pth checkpoint
--input	required	Image file, folder, or video
--output	auto	Output path (auto-named if omitted)
--input-size	384	Shorter-side length for model input. Aspect ratio is preserved; the longer side scales proportionally. Rounded to the nearest multiple of 32 (e.g. 384, 512, 768)
--fp16	off	FP16 precision (CUDA only)
--compile	off	torch.compile for faster steady-state throughput
--npu	off	Use NPU-compatible upsampling — required when loading zipdepth_base_npu.pth
--save-raw	off	Also save depth map as .npy
--no-colormap	off	Skip colorized JPEG — use with --save-raw for raw depth only
--output-size	model input	Output video height in pixels (e.g. 1080 for 1080p)
--max-frames	all	Limit number of video frames to process

Batch and video inference use an asynchronous pipeline: GPU inference and CPU colormap/write are overlapped, keeping the GPU continuously busy. Timing is measured with CUDA events.

More examples

# Raw depth maps only — maximum speed, no visualization
python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/imgs/ \
  --no-colormap --save-raw

# FP16 + torch.compile — highest throughput
python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/imgs/ \
  --fp16 --compile

# 4K video → 1080p output
python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/clip.mp4 \
  --output-size 1080

---

📦 Export

ONNX

# GPU checkpoint (default)
python scripts/export.py \
  --ckpt checkpoints/zipdepth_base.pth \
  --format onnx \
  --height 384 --width 384

# NPU / mobile checkpoint — add --npu
python scripts/export.py \
  --ckpt checkpoints/zipdepth_base_npu.pth \
  --format onnx \
  --height 384 --width 384 --npu

Install onnxsim for automatic graph simplification (applied transparently if available).

TorchScript

# Traced
python scripts/export.py \
  --ckpt checkpoints/zipdepth_base.pth \
  --format torchscript

# Frozen — smaller and faster on CPU
python scripts/export.py \
  --ckpt checkpoints/zipdepth_base.pth \
  --format torchscript-frozen

The output is saved next to the checkpoint by default. Pass --output to override the path.

---

📱 Mobile / Edge Deployment

Both checkpoints can be exported and run on-device — you're free to try either, but the two upsampling variants are not equally portable:

• zipdepth_base_npu.pth — recommended for mobile / edge / NPU. Its unfold-free convex upsampling is built from operators that convert cleanly across mobile runtimes. Export with --npu.
• zipdepth_base.pth relies on torch.nn.Unfold + pixel_shuffle, which several mobile runtimes lower poorly (or not at all). Great on GPU/server, less reliable on-device.

Typical path: export to ONNX (see above), then convert to your target runtime — ONNX Runtime Mobile, CoreML (iOS), TFLite (Android), or NCNN. Starting from the NPU checkpoint maximizes the chance of a clean, fully-supported conversion.

On-device latency and operator-level profiling across hardware are reported in the Deployment Profiling section of the supplementary material.

---

📈 Benchmark

Measures parameters, GFLOPs, and latency across backends with IQR-filtered statistics.

python scripts/benchmark.py --height 384 --width 384
python scripts/benchmark.py --height 384 --width 384 --fp16

Representative latency on an RTX 3090 (ZipDepth-base, 384×384, PyTorch 2.4.1+cu121, CUDA 12.1), from the paper's deployment profiling — median over 200 forward passes (20 warm-up):

  Backend                       Precision   Latency     FPS   Speedup
  ─────────────────────────────────────────────────────────────────────
  PyTorch Eager                 FP32         3.9 ms     255    1.0×
  PyTorch Fused                 FP32         2.5 ms     389    1.5×
  PyTorch Fused                 FP16         3.2 ms     307    1.2×
  torch.compile (max-autotune)  FP16         0.9 ms    1117    4.4×
  TensorRT (static)             FP16         0.8 ms    1317    4.9×

Fused = Conv-BN + reparameterizable branch fusion via fuse_for_inference(). torch.compile uses mode="max-autotune"; TensorRT is a static-shape FP16 engine (measured separately from benchmark.py).

---

🧪 Evaluation

We follow the zero-shot protocol of Marigold: predictions are aligned to the ground truth with a least-squares scale and shift, then standard depth metrics are computed. Please refer to Marigold for downloading and preparing the benchmark datasets.

Evaluate a checkpoint on any supported benchmark with a single command:

python scripts/eval.py \
  --dataset nyuv2 \
  --data_dir /path/to/NYUv2/test \
  --checkpoint checkpoints/zipdepth_base.pth

--dataset	Benchmark	Expected layout
nyuv2	NYUv2	{scene}/rgb_*.png + depth_*.png
kitti	KITTI (Eigen)	{date}/{drive}/image_02/data/*.png + proj_depth/groundtruth/image_02/*.png
eth3d	ETH3D	depth/{scene}/... + images/{scene}/...
scannet	ScanNet	{scene}/color/*.jpg + depth/*.png
diode	DIODE	{indoors,outdoor}/scene_*/scan_*/*.png + *_depth.npy

The per-dataset depth range, KITTI crop, and Eigen mask are applied automatically. Metrics are printed and saved to eval_results/<dataset>/accuracy_<dataset>.json, with per-sample values in per_sample.csv. Add --fp16 for half-precision inference, or --npu when evaluating the zipdepth_base_npu.pth checkpoint.

---

🎓 Training

Training Data

ZipDepth was trained via knowledge distillation using pseudo depth maps generated by Depth Anything V2 Large. The training set spans 17 domains and contains approximately 14.1 million RGB–depth pairs:

ACDC · ADE20K · bdd100k · Cityscapes · COCO · DrivingStereo · Flickr1024 · Gated2Depth · GoogleLandmarks · HRWSI · HoloPix50K · Mapillary · MegaDepth · Object365 · OpenImagesv7 · SA-1B · Trans10K

The exact file list used for training — 14,069,951 images across these 17 domains — is released as a downloadable asset:

📄 training_files.txt.gz (71 MB compressed)

Each line is domain<TAB>relative_path. This is the file list only — the RGB images must be obtained from the original public datasets listed above, under their respective licenses.

The proxy depth labels are likewise not distributed: they can be regenerated by running the official Depth Anything V2 Large inference code on the RGB images, exactly as done in the paper.

Dataset Index

The dataloader expects a JSON index of RGB–depth pairs. To build one from your own data, organize images into parallel RGB and depth directories with matching relative paths, then run:

# Single domain
python scripts/prepare_index.py build \
    --domains MyDataset /path/to/rgb /path/to/depth \
    --output dataset_index.json

# Multiple domains via a YAML config
python scripts/prepare_index.py build \
    --config domains.yaml --output dataset_index.json

YAML config format:

MyDataset:
    rgb:   /path/to/rgb
    depth: /path/to/depth
AnotherSet:
    rgb:   /path/to/rgb2
    depth: /path/to/depth2

The depth maps can be PNG (uint16) or .npy/.npz files. Before training, convert the JSON index to numpy memmap format (much faster I/O at millions of samples):

python scripts/prepare_index.py convert --input dataset_index.json

Or do both in one shot with --convert:

python scripts/prepare_index.py build \
    --domains MyDataset /path/to/rgb /path/to/depth \
    --output dataset_index.json --convert

Set index_file in configs/default.json to the JSON path — the dataloader auto-detects the converted .npy files.

Launch

Edit configs/default.json to set your dataset path and hyperparameters, then launch:

# Single GPU
python scripts/train.py --config configs/default.json

# Multi-GPU with DDP (e.g. 2 GPUs)
torchrun --nproc_per_node=2 scripts/train.py --config configs/default.json

# Resume from a checkpoint
python scripts/train.py \
  --config configs/default.json \
  --resume checkpoints/base_384x384/epoch_3.pth

Key configuration fields:

{
  "model":    { "variant": "base" },
  "data":     { "index_file": "/path/to/dataset_index.json", "height": 384, "width": 384 },
  "training": { "epochs": 5, "batch_size": 96 },
  "optimizer":{ "lr": 1e-3, "weight_decay": 0.05 },
  "amp":      { "enabled": true, "dtype": "bfloat16" }
}

Training uses a scale-and-shift invariant loss with gradient regularization, AdamW with OneCycleLR scheduling, and bfloat16 mixed precision.

---

🙏 Acknowledgements

We thank the authors of Marigold and Depth Anything V2 for their excellent work and for releasing code and evaluation protocols that made this research possible.

---

📝 Citation

@inproceedings{tosi2026zipdepth,
  title     = {ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device},
  author    = {Tosi, Fabio and Bartolomei, Luca and Poggi, Matteo and Mattoccia, Stefano},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}

---

📧 Contact

For questions about the paper or the code, feel free to reach out:

• Fabio Tosi — fabio.tosi5@unibo.it
• Luca Bartolomei — luca.bartolomei5@unibo.it
• Matteo Poggi — m.poggi@unibo.it
• Stefano Mattoccia — stefano.mattoccia@unibo.it