MVS-GS: High-Quality 3D Gaussian Splatting Mapping via Online Multi-View Stereo
This study addresses the challenge of online 3D model generation for neural rendering using an RGB image stream. Previous research has tackled this issue by incorporating Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (3DGS) as scene representations within dense SLAM methods. However, most s...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11050405/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849427927950688256 |
|---|---|
| author | Byeonggwon Lee Junkyu Park Khang Truong Giang Sungho Jo Soohwan Song |
| author_facet | Byeonggwon Lee Junkyu Park Khang Truong Giang Sungho Jo Soohwan Song |
| author_sort | Byeonggwon Lee |
| collection | DOAJ |
| description | This study addresses the challenge of online 3D model generation for neural rendering using an RGB image stream. Previous research has tackled this issue by incorporating Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (3DGS) as scene representations within dense SLAM methods. However, most studies focus primarily on estimating coarse 3D scenes rather than achieving detailed reconstructions. Moreover, depth estimation based solely on images is often ambiguous, resulting in low-quality 3D models that lead to inaccurate renderings. To overcome these limitations, we propose a novel framework for high-quality 3DGS modeling that leverages an online multi-view stereo (MVS) approach. Our method estimates MVS depth using sequential frames from a local time window and applies comprehensive depth refinement techniques to filter out outliers. The refinement method produces temporally consistent depths by checking sequential geometric consistency, enabling accurate initialization of Gaussians in 3DGS. Furthermore, we introduce a parallelized backend module that optimizes the 3DGS model efficiently, ensuring timely updates with each new keyframe. Experimental results demonstrate that our method outperforms state-of-the-art dense SLAM methods, achieving an average PSNR improvement of approximately 2 dB on indoor scenes. Moreover, our method reliably produces consistent 3D models in complex outdoor scenes, where existing methods often fail due to tracking errors and depth noise. It also reconstructs large-scale aerial scenes effectively, achieving an average PSNR gain of about 10.28 dB over existing methods. |
| format | Article |
| id | doaj-art-60bd7a57998a49a1a472720947e73518 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-60bd7a57998a49a1a472720947e735182025-08-20T03:28:52ZengIEEEIEEE Access2169-35362025-01-011311310.1109/ACCESS.2025.358315611050405MVS-GS: High-Quality 3D Gaussian Splatting Mapping via Online Multi-View StereoByeonggwon Lee0https://orcid.org/0009-0006-7065-4263Junkyu Park1Khang Truong Giang2https://orcid.org/0000-0002-6709-7885Sungho Jo3https://orcid.org/0000-0002-7618-362XSoohwan Song4https://orcid.org/0000-0003-2145-1161Department of Computer Science and Artificial Intelligence, Dongguk University, Seoul, Republic of KoreaDepartment of Computer Science and Artificial Intelligence, Dongguk University, Seoul, Republic of Korea42dot, Seongnam-si, Gyeonggi-do, Republic of KoreaSchool of Computing, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of KoreaDepartment of Computer Science and Artificial Intelligence, Dongguk University, Seoul, Republic of KoreaThis study addresses the challenge of online 3D model generation for neural rendering using an RGB image stream. Previous research has tackled this issue by incorporating Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (3DGS) as scene representations within dense SLAM methods. However, most studies focus primarily on estimating coarse 3D scenes rather than achieving detailed reconstructions. Moreover, depth estimation based solely on images is often ambiguous, resulting in low-quality 3D models that lead to inaccurate renderings. To overcome these limitations, we propose a novel framework for high-quality 3DGS modeling that leverages an online multi-view stereo (MVS) approach. Our method estimates MVS depth using sequential frames from a local time window and applies comprehensive depth refinement techniques to filter out outliers. The refinement method produces temporally consistent depths by checking sequential geometric consistency, enabling accurate initialization of Gaussians in 3DGS. Furthermore, we introduce a parallelized backend module that optimizes the 3DGS model efficiently, ensuring timely updates with each new keyframe. Experimental results demonstrate that our method outperforms state-of-the-art dense SLAM methods, achieving an average PSNR improvement of approximately 2 dB on indoor scenes. Moreover, our method reliably produces consistent 3D models in complex outdoor scenes, where existing methods often fail due to tracking errors and depth noise. It also reconstructs large-scale aerial scenes effectively, achieving an average PSNR gain of about 10.28 dB over existing methods.https://ieeexplore.ieee.org/document/11050405/Online multi-view stereo3D Gaussian splattingneural renderingdense SLAM3D modelingdepth estimation |
| spellingShingle | Byeonggwon Lee Junkyu Park Khang Truong Giang Sungho Jo Soohwan Song MVS-GS: High-Quality 3D Gaussian Splatting Mapping via Online Multi-View Stereo IEEE Access Online multi-view stereo 3D Gaussian splatting neural rendering dense SLAM 3D modeling depth estimation |
| title | MVS-GS: High-Quality 3D Gaussian Splatting Mapping via Online Multi-View Stereo |
| title_full | MVS-GS: High-Quality 3D Gaussian Splatting Mapping via Online Multi-View Stereo |
| title_fullStr | MVS-GS: High-Quality 3D Gaussian Splatting Mapping via Online Multi-View Stereo |
| title_full_unstemmed | MVS-GS: High-Quality 3D Gaussian Splatting Mapping via Online Multi-View Stereo |
| title_short | MVS-GS: High-Quality 3D Gaussian Splatting Mapping via Online Multi-View Stereo |
| title_sort | mvs gs high quality 3d gaussian splatting mapping via online multi view stereo |
| topic | Online multi-view stereo 3D Gaussian splatting neural rendering dense SLAM 3D modeling depth estimation |
| url | https://ieeexplore.ieee.org/document/11050405/ |
| work_keys_str_mv | AT byeonggwonlee mvsgshighquality3dgaussiansplattingmappingviaonlinemultiviewstereo AT junkyupark mvsgshighquality3dgaussiansplattingmappingviaonlinemultiviewstereo AT khangtruonggiang mvsgshighquality3dgaussiansplattingmappingviaonlinemultiviewstereo AT sunghojo mvsgshighquality3dgaussiansplattingmappingviaonlinemultiviewstereo AT soohwansong mvsgshighquality3dgaussiansplattingmappingviaonlinemultiviewstereo |