High-Fidelity Depth Map Reconstruction System With RGB-Guided Super Resolution CNN and Cross-Calibrated Chaos LiDAR
High-fidelity 3D models are essential for immersive virtual and augmented reality (VR/AR) applications. However, the performance of current 3D recording devices is limited in several scenarios, such as dim light environments, long-distance measurements, and large-scale objects. Therefore, their appl...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10849544/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832576800296796160 |
|---|---|
| author | Yu-Chun Ding Chia-Yu Chang Pei-Rong Li Chao-Tsung Huang Yung-Chen Lin Tsung Chen Wei-Lun Lin Cheng-Ting Lee Fan-Yi Lin Yuan-Hao Huang |
| author_facet | Yu-Chun Ding Chia-Yu Chang Pei-Rong Li Chao-Tsung Huang Yung-Chen Lin Tsung Chen Wei-Lun Lin Cheng-Ting Lee Fan-Yi Lin Yuan-Hao Huang |
| author_sort | Yu-Chun Ding |
| collection | DOAJ |
| description | High-fidelity 3D models are essential for immersive virtual and augmented reality (VR/AR) applications. However, the performance of current 3D recording devices is limited in several scenarios, such as dim light environments, long-distance measurements, and large-scale objects. Therefore, their applicability to indoor scenes is hindered. In this work, we propose a depth map reconstruction system that integrates an RGB-guided depth map super-resolution convolutional neural network (CNN) into a stand-alone Chaos LiDAR depth sensor. This system provides highly accurate depth estimates in various scenarios, particularly for indoor scenes with dim lighting or long distances ranging from 4 m to 6 m. We address two design challenges to maximize the quality of the reconstructed depth map of the system. First, the misalignment across RGB-depth sensors is addressed using a two-stage calibration pipeline. Second, the lack of large-scale real-world LiDAR datasets is addressed by generating a large-scale synthetic dataset and adopting transfer learning. Experimental results show that our proposed system significantly outperforms the commercial RGB-D recording device RealSense D435i in terms of subjective visual perception, precision, and density of depth estimates, making it a promising solution for general indoor scene recording. |
| format | Article |
| id | doaj-art-fb89c3d14640454aba7d737adf8d5174 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-fb89c3d14640454aba7d737adf8d51742025-01-31T00:01:28ZengIEEEIEEE Access2169-35362025-01-0113191181913110.1109/ACCESS.2025.353262110849544High-Fidelity Depth Map Reconstruction System With RGB-Guided Super Resolution CNN and Cross-Calibrated Chaos LiDARYu-Chun Ding0https://orcid.org/0000-0001-9004-3597Chia-Yu Chang1Pei-Rong Li2Chao-Tsung Huang3https://orcid.org/0000-0002-9173-520XYung-Chen Lin4Tsung Chen5https://orcid.org/0009-0008-9492-3784Wei-Lun Lin6https://orcid.org/0009-0003-5187-0495Cheng-Ting Lee7Fan-Yi Lin8https://orcid.org/0000-0003-2160-9715Yuan-Hao Huang9https://orcid.org/0000-0001-6781-7312Department of Electrical Engineering, National Tsing Hua University, Hsinchu, TaiwanDepartment of Electrical Engineering, National Tsing Hua University, Hsinchu, TaiwanDepartment of Electrical Engineering, National Tsing Hua University, Hsinchu, TaiwanDepartment of Electrical Engineering, National Tsing Hua University, Hsinchu, TaiwanDepartment of Electrical Engineering, National Tsing Hua University, Hsinchu, TaiwanDepartment of Electrical Engineering, Institute of Communications Engineering, National Tsing Hua University, Hsinchu, TaiwanDepartment of Electrical Engineering, National Tsing Hua University, Hsinchu, TaiwanDepartment of Electrical Engineering, Institute of Photonics Technologies, National Tsing Hua University, Hsinchu, TaiwanDepartment of Electrical Engineering, Institute of Photonics Technologies, National Tsing Hua University, Hsinchu, TaiwanDepartment of Electrical Engineering, Institute of Communications Engineering, National Tsing Hua University, Hsinchu, TaiwanHigh-fidelity 3D models are essential for immersive virtual and augmented reality (VR/AR) applications. However, the performance of current 3D recording devices is limited in several scenarios, such as dim light environments, long-distance measurements, and large-scale objects. Therefore, their applicability to indoor scenes is hindered. In this work, we propose a depth map reconstruction system that integrates an RGB-guided depth map super-resolution convolutional neural network (CNN) into a stand-alone Chaos LiDAR depth sensor. This system provides highly accurate depth estimates in various scenarios, particularly for indoor scenes with dim lighting or long distances ranging from 4 m to 6 m. We address two design challenges to maximize the quality of the reconstructed depth map of the system. First, the misalignment across RGB-depth sensors is addressed using a two-stage calibration pipeline. Second, the lack of large-scale real-world LiDAR datasets is addressed by generating a large-scale synthetic dataset and adopting transfer learning. Experimental results show that our proposed system significantly outperforms the commercial RGB-D recording device RealSense D435i in terms of subjective visual perception, precision, and density of depth estimates, making it a promising solution for general indoor scene recording.https://ieeexplore.ieee.org/document/10849544/Depth map super-resolutionChaos LiDARdepth sensing |
| spellingShingle | Yu-Chun Ding Chia-Yu Chang Pei-Rong Li Chao-Tsung Huang Yung-Chen Lin Tsung Chen Wei-Lun Lin Cheng-Ting Lee Fan-Yi Lin Yuan-Hao Huang High-Fidelity Depth Map Reconstruction System With RGB-Guided Super Resolution CNN and Cross-Calibrated Chaos LiDAR IEEE Access Depth map super-resolution Chaos LiDAR depth sensing |
| title | High-Fidelity Depth Map Reconstruction System With RGB-Guided Super Resolution CNN and Cross-Calibrated Chaos LiDAR |
| title_full | High-Fidelity Depth Map Reconstruction System With RGB-Guided Super Resolution CNN and Cross-Calibrated Chaos LiDAR |
| title_fullStr | High-Fidelity Depth Map Reconstruction System With RGB-Guided Super Resolution CNN and Cross-Calibrated Chaos LiDAR |
| title_full_unstemmed | High-Fidelity Depth Map Reconstruction System With RGB-Guided Super Resolution CNN and Cross-Calibrated Chaos LiDAR |
| title_short | High-Fidelity Depth Map Reconstruction System With RGB-Guided Super Resolution CNN and Cross-Calibrated Chaos LiDAR |
| title_sort | high fidelity depth map reconstruction system with rgb guided super resolution cnn and cross calibrated chaos lidar |
| topic | Depth map super-resolution Chaos LiDAR depth sensing |
| url | https://ieeexplore.ieee.org/document/10849544/ |
| work_keys_str_mv | AT yuchunding highfidelitydepthmapreconstructionsystemwithrgbguidedsuperresolutioncnnandcrosscalibratedchaoslidar AT chiayuchang highfidelitydepthmapreconstructionsystemwithrgbguidedsuperresolutioncnnandcrosscalibratedchaoslidar AT peirongli highfidelitydepthmapreconstructionsystemwithrgbguidedsuperresolutioncnnandcrosscalibratedchaoslidar AT chaotsunghuang highfidelitydepthmapreconstructionsystemwithrgbguidedsuperresolutioncnnandcrosscalibratedchaoslidar AT yungchenlin highfidelitydepthmapreconstructionsystemwithrgbguidedsuperresolutioncnnandcrosscalibratedchaoslidar AT tsungchen highfidelitydepthmapreconstructionsystemwithrgbguidedsuperresolutioncnnandcrosscalibratedchaoslidar AT weilunlin highfidelitydepthmapreconstructionsystemwithrgbguidedsuperresolutioncnnandcrosscalibratedchaoslidar AT chengtinglee highfidelitydepthmapreconstructionsystemwithrgbguidedsuperresolutioncnnandcrosscalibratedchaoslidar AT fanyilin highfidelitydepthmapreconstructionsystemwithrgbguidedsuperresolutioncnnandcrosscalibratedchaoslidar AT yuanhaohuang highfidelitydepthmapreconstructionsystemwithrgbguidedsuperresolutioncnnandcrosscalibratedchaoslidar |