aSDF: Reconstructing Real-Time Signed Distance Fields with Enhanced Attention

We present aSDF, a continual learning system based on attention for real-time signed distance field reconstruction. Given a stream of posed monocular RGB images from a moving camera, it trains a randomly initialized network to map input 3D coordinates to an approximated signed distance. The model is based on a transformer encoder enhanced with a hash table which acts as a cache, significantly improving the learning time, a crucial component in a continual learning system. In contrast to prior works, our model further improves the compact representation, while maintaining a decent accuracy in terms of the reconstructed signed distances. By employing self-supervised losses with a bigger penalization in the case of wrong prediction, the network is able to improve the accuracy of unobserved regions by filling out with a more plausible signed value. In evaluation to other methods on indoor synthetic and real datasets of indoor rooms, aSDF produces competitive reconstruction with previous state-of-the-art depth based methods and better approximation of gradients, while converging much faster due to the hashing component.