TC-Depth

Project Page | arXiv | Workshop

Official Repository for

Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation

Patrick Ruhkamp*, Daoyi Gao*, Hanzhi Chen*, Nassir Navab, Benjamin Busam - 3DV, 2021.

Spatial-Temporal Attention through Self-Supervised Geometric Guidance

Patrick Ruhkamp*, Daoyi Gao*, Hanzhi Chen*, Nassir Navab, Benjamin Busam - ICCV Workshop on Self-supervised Learning for Next-Generation Industry-level Autonomous Driving, 2021.

* equal contribution

🤓 TL;DR

• Current SOTA in self-supervised monocular depth estimation achievies highly accurate depth predictions, but suffer from inconsistencies across temporal frames
• Our novel Spatial-Temporal Attention mechanism with Geometric Guidance improves consistency while maintaining accuracy
• The Temporal Consistency Metric (TCM) is a quantitative measure to evaluate the consistency between temporal predictions in 3D

News

• Evaluation code for TCM available (02.12.2021)
• Release training code (tbd)

✏️ 📄 Citation

Please consider to cite our paper:

@inproceedings{ruhkamp2021attention,
    title = {Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation},
    author = {Patrick Ruhkamp and
              Daoyi Geo and
              Hanzhi Chen and
              Nassir Navab and
              Benjamin Busam},
    booktitle = {IEEE International Conference on 3D Vision (3DV)},
    year = {2021},
    month = {December}
}

Qualitative Results

Spatial-Temporal Attention

Temporal Consistency Metric (TCM)

GT for TCM

3 Frames | 5 Frames | 7 Frames