Get the latest tech news
Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting
Visual Geometry Group, University of Oxford To hallucinate scenes beyond known regions and lift images generated by 2D-based models into three dimensions, current 3D scene generation methods rely on monocular depth estimation networks. For this task, it is crucial to seamlessly integrate the newly hallucinated regions into the existing scene representation.
@inproceedings{ engstler2024invisible, title={Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting} author={Paul Engstler and Andrea Vedaldi and Iro Laina and Christian Rupprecht} year={2024} booktitle={Arxiv} } Simple global scale-and-shift operations to the predicted depth map, as used by previous methods, might lead to discontinuities between the scene and its hallucinated extension. We thus introduce a novel depth completion model, trained via teacher distillation and self-training to learn the 3D fusion process, resulting in improved geometric coherence of the scene.
Or read this on Hacker News
