ThermalGS: Dynamic 3D Thermal Reconstruction with Gaussian Splatting
Thermal infrared (TIR) images capture temperature in a non-invasive manner, making them valuable for generating 3D models that reflect the spatial distribution of thermal properties within a scene. Current TIR image-based 3D reconstruction methods primarily focus on static conditions, which only cap...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/17/2/335 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832587569562386432 |
|---|---|
| author | Yuxiang Liu Xi Chen Shen Yan Zeyu Cui Huaxin Xiao Yu Liu Maojun Zhang |
| author_facet | Yuxiang Liu Xi Chen Shen Yan Zeyu Cui Huaxin Xiao Yu Liu Maojun Zhang |
| author_sort | Yuxiang Liu |
| collection | DOAJ |
| description | Thermal infrared (TIR) images capture temperature in a non-invasive manner, making them valuable for generating 3D models that reflect the spatial distribution of thermal properties within a scene. Current TIR image-based 3D reconstruction methods primarily focus on static conditions, which only capture the spatial distribution of thermal radiation but lack the ability to represent its temporal dynamics. The absence of dedicated datasets and effective methods for dynamic 3D representation are two key challenges that hinder progress in this field. To address these challenges, we propose a novel dynamic thermal 3D reconstruction method, named ThermalGS, based on 3D Gaussian Splatting (3DGS). ThermalGS employs a data-driven approach to directly learn both scene structure and dynamic thermal representation, using RGB and TIR images as input. The position, orientation, and scale of Gaussian primitives are guided by the RGB mesh. We introduce feature encoding and embedding networks to integrate semantic and temporal information into the Gaussian primitives, allowing them to capture dynamic thermal radiation characteristics. Moreover, we construct the Thermal Scene Day-and-Night (TSDN) dataset, which includes multi-view, high-resolution aerial RGB reference images and TIR images captured at five different times throughout the day and night, providing a benchmark for dynamic thermal 3D reconstruction tasks. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on the TSDN dataset, with an average absolute temperature error of 1 °C and the ability to predict surface temperature variations over time. |
| format | Article |
| id | doaj-art-55aad5a5dbc547d3b2441392b948958c |
| institution | Kabale University |
| issn | 2072-4292 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj-art-55aad5a5dbc547d3b2441392b948958c2025-01-24T13:48:10ZengMDPI AGRemote Sensing2072-42922025-01-0117233510.3390/rs17020335ThermalGS: Dynamic 3D Thermal Reconstruction with Gaussian SplattingYuxiang Liu0Xi Chen1Shen Yan2Zeyu Cui3Huaxin Xiao4Yu Liu5Maojun Zhang6College of Systems and Engineering, National University of Defense Technology, Changsha 410073, ChinaCollege of Systems and Engineering, National University of Defense Technology, Changsha 410073, ChinaCollege of Systems and Engineering, National University of Defense Technology, Changsha 410073, ChinaCollege of Systems and Engineering, National University of Defense Technology, Changsha 410073, ChinaCollege of Systems and Engineering, National University of Defense Technology, Changsha 410073, ChinaCollege of Systems and Engineering, National University of Defense Technology, Changsha 410073, ChinaCollege of Systems and Engineering, National University of Defense Technology, Changsha 410073, ChinaThermal infrared (TIR) images capture temperature in a non-invasive manner, making them valuable for generating 3D models that reflect the spatial distribution of thermal properties within a scene. Current TIR image-based 3D reconstruction methods primarily focus on static conditions, which only capture the spatial distribution of thermal radiation but lack the ability to represent its temporal dynamics. The absence of dedicated datasets and effective methods for dynamic 3D representation are two key challenges that hinder progress in this field. To address these challenges, we propose a novel dynamic thermal 3D reconstruction method, named ThermalGS, based on 3D Gaussian Splatting (3DGS). ThermalGS employs a data-driven approach to directly learn both scene structure and dynamic thermal representation, using RGB and TIR images as input. The position, orientation, and scale of Gaussian primitives are guided by the RGB mesh. We introduce feature encoding and embedding networks to integrate semantic and temporal information into the Gaussian primitives, allowing them to capture dynamic thermal radiation characteristics. Moreover, we construct the Thermal Scene Day-and-Night (TSDN) dataset, which includes multi-view, high-resolution aerial RGB reference images and TIR images captured at five different times throughout the day and night, providing a benchmark for dynamic thermal 3D reconstruction tasks. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on the TSDN dataset, with an average absolute temperature error of 1 °C and the ability to predict surface temperature variations over time.https://www.mdpi.com/2072-4292/17/2/3353D thermal reconstruction3DGSthermal image datasetdynamic scene reconstruction |
| spellingShingle | Yuxiang Liu Xi Chen Shen Yan Zeyu Cui Huaxin Xiao Yu Liu Maojun Zhang ThermalGS: Dynamic 3D Thermal Reconstruction with Gaussian Splatting Remote Sensing 3D thermal reconstruction 3DGS thermal image dataset dynamic scene reconstruction |
| title | ThermalGS: Dynamic 3D Thermal Reconstruction with Gaussian Splatting |
| title_full | ThermalGS: Dynamic 3D Thermal Reconstruction with Gaussian Splatting |
| title_fullStr | ThermalGS: Dynamic 3D Thermal Reconstruction with Gaussian Splatting |
| title_full_unstemmed | ThermalGS: Dynamic 3D Thermal Reconstruction with Gaussian Splatting |
| title_short | ThermalGS: Dynamic 3D Thermal Reconstruction with Gaussian Splatting |
| title_sort | thermalgs dynamic 3d thermal reconstruction with gaussian splatting |
| topic | 3D thermal reconstruction 3DGS thermal image dataset dynamic scene reconstruction |
| url | https://www.mdpi.com/2072-4292/17/2/335 |
| work_keys_str_mv | AT yuxiangliu thermalgsdynamic3dthermalreconstructionwithgaussiansplatting AT xichen thermalgsdynamic3dthermalreconstructionwithgaussiansplatting AT shenyan thermalgsdynamic3dthermalreconstructionwithgaussiansplatting AT zeyucui thermalgsdynamic3dthermalreconstructionwithgaussiansplatting AT huaxinxiao thermalgsdynamic3dthermalreconstructionwithgaussiansplatting AT yuliu thermalgsdynamic3dthermalreconstructionwithgaussiansplatting AT maojunzhang thermalgsdynamic3dthermalreconstructionwithgaussiansplatting |