Weakly-Supervised Single-view Dense 3D Point Cloud Reconstruction via Differentiable Renderer

• 作 者: Peng Jin Shaoli Liu Jianhua Liu Hao Huang Linlin Yang Michael Weinmann Reinhard Klein
• 机 构: School of Mechanical Engineering,Beijing Institute of Technology,Bei-jing 100081,China;Institute of
• 出 处: 中国机械工程学报
• 发表日期: 2021年5期
• 关键词: Point clouds reconstruction Differentiable renderer Neural networks Single-view

In recent years,addressing ill-posed problems by leveraging prior knowledge contained in databases on learning techniques has gained much attention.In this paper,we focus on complete three-dimensional(3D)point cloud reconstruction based on a single red-green-blue(RGB)image,a task that cannot be approached using classical recon-struction techniques.For this purpose,we used an encoder-decoder framework to encode the RGB information in latent space,and to predict the 3D structure of the considered object from different viewpoints.The individual predic-tions are combined to yield a common representation that is used in a module combining camera pose estimation and rendering,thereby achieving differentiability with respect to imaging process and the camera pose,and optimi-zation of the two-dimensional prediction error of novel viewpoints.Thus,our method allows end-to-end training and does not require supervision based on additional ground-truth(GT)mask annotations or ground-truth camera pose annotations.Our evaluation of synthetic and real-world data demonstrates the robustness of our approach to appear-ance changes and self-occlusions,through outperformance of current state-of-the-art methods in terms of accuracy,density,and model completeness.