Robust and Efficient 3D Gaussian Splatting for Diagnostic Imaging

Abstract

3D Gaussian Splatting (3DGS) has emerged as a popular technique for neural rendering. Use of simple Gaussian primitives that enable real-time rendering, faster training, and high visual fidelity makes its use compelling over alternatives like NeRFs. However, despite its success in natural image settings, 3DGS encounters notable rendering challenges in medical imaging. Our analysis reveals that the error characteristics of sparse reconstruction pipelines (e.g., COLMAP), which serve input to the 3DGS pipeline, differ significantly between medical and natural environments. This discrepancy leads to degraded rendering performance when noise propagates from sparsely reconstructed inputs. To address this issue, we introduce a novel opacity-based regularization strategy for 3DGS that yields two key benefits: (1) enhanced rendering quality through suppression of noisy Gaussian components, and (2) substantial reduction in the number of Gaussian primitives required, improving rendering efficiency. Experiments on two medical datasets validate the effectiveness of our approach. On the C3VD dataset, our method achieves an average PSNR improvement of 0.42 dB and reduces the number of Gaussians by 14,137 per sequence. On the EndoMap-per dataset, we observe an average PSNR gain of 1.60 dB with a reduction of 91,522 Gaussians per sequence.

Figure 1: Samples of the Robust 3D Gaussian Splatting framework.