Benchmarking Learnable Mesh and Texture Representations for Immersive Digital Twins

Abstract

Neural radiance fields (NeRF) and 3D Gaussian splatting (3DGS) use volumetric scene representations to achieve impressive visual results in the field of novel-view synthesis. However, traditional 3D pipelines are dominated by textured meshes, supported by hardware assisted rendering and a huge software ecosystem. We show that mesh-based workflows can also profit from those novel reconstruction methods by evaluating mesh reconstruction algorithms paired with view-dependent textures in terms of texture sharpness, surface accuracy and real-time rendering performance. For that purpose, we employ a modular 3D reconstruction pipeline and use it to benchmark not only publicly available data sets, but additionally four new high-quality data sets of our own. These data sets capture different objects containing both reflective and uniform surface characteristics.