Softmesh: Learning Probabilistic Mesh Connectivity via Image Supervision

In this work we introduce Softmesh, a fully differentiable pipeline to transform a 3D point cloud into a probabilistic mesh representation that allows us to directly render 2D images. We use this pipeline to learn point connectivity from only 2D rendering supervision, reducing the supervision requirements for mesh-based representations.We evaluate our approach in a set of rendering tasks, including silhouette, normal, and depth rendering on both rigid and non-rigid objects. We introduce transfer learning approaches to handle the diversity of the task requirements, and also explore the potential of learning across categories. We demonstrate that Softmesh achieves competitive performance even against methods trained with full mesh supervision.