Learning signed distance functions (SDFs) from 3D point clouds is an important task in 3D computer vision. However, without ground truth signed distances, point normals or clean point clouds, current methods still struggle from learning SDFs from noisy point clouds. To overcome this challenge, we propose to learn SDFs via a noise to noise mapping, which does not require any clean point cloud or ground truth supervision for training. Our novelty lies in the noise to noise mapping which can infer a highly accurate SDF of a single object or scene from its multiple or even single noisy point cloud observations. Our novel learning manner is supported by modern Lidar systems which capture multiple noisy observations per second. We achieve this by a novel loss which enables statistical reasoning on point clouds and maintains geometric consistency although point clouds are irregular, unordered and have no point correspondence among noisy observations. Our evaluation under the widely used benchmarks demonstrates our superiority over the state-of-the-art methods in surface reconstruction, point cloud denoising and upsampling.
Overview of our method. Given \(N\) corrupted observations \(S=\{{N}_i|i\in[1,N],N\ge 1\}\) of an uncorrupted 3D shape or scene \({S}\), we aim to learn SDFs \(f\) of \({S}\) from \(S\) without ground truth signed distances, point normals, or clean point clouds. Here, \({N}_i\) is a noisy point cloud. SDFs \(f\) predicts a signed distance \(d\) for an arbitrary query location \({q}\in \mathbb{R}^{1\times 3}\) around \({S}\), such that \(d=f({q},{c})\), where \({c}\) is a condition denoting \({S}\). We train a neural network parameterized by \({\theta}\) to learn \(f\), which we denote as \(f_{{\theta}}\). After training, we can further leverage the learned \(f_{{\theta}}\) for surface reconstruction, point cloud denoising, and point cloud upsampling.
Demonstration of denoising on real scans (e.g. Paris-rue-Madame scenes)
@ARTICLE{FastN2N,
author={Zhou, Junsheng and Ma, Baorui and Liu, Yu-Shen and Han, Zhizhong},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
title={Fast Learning of Signed Distance Functions From Noisy Point Clouds Via Noise to Noise Mapping},
year={2024},
volume={},
number={},
pages={1-18},
keywords={Point cloud compression;Noise;Noise measurement;Three-dimensional displays;Noise reduction;Image reconstruction;Surface reconstruction;Fast learning;noise to noise mapping;point cloud denoising;signed distance functions;surface reconstruction},
doi={10.1109/TPAMI.2024.3416068}}
}
@inproceedings{BaoruiNoise2NoiseMapping,
title = {Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping},
author = {Baorui Ma and Yu-Shen Liu and Zhizhong Han},
booktitle = {International Conference on Machine Learning (ICML)},
year = {2023}
}