Investigating the effects of hyperparameter sensitivity on machine learning algorithms for PV forecasting
Machine Learning (ML) models have been introduced in the past, and users have debated whether to tune the hyperparameters of the models. This study investigates the effects of tuning the hyperparameters of the ML models and summarizes the models that are most sensitive to hyperparameter tuning. This...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
EDP Sciences
2025-01-01
|
| Series: | E3S Web of Conferences |
| Online Access: | https://www.e3s-conferences.org/articles/e3sconf/pdf/2025/12/e3sconf_aere2025_01002.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832098456631508992 |
|---|---|
| author | Ehtsham Muhammad Rotilio Marianna Cucchiella Federica Di Giovanni Gianni Schettini Domenico |
| author_facet | Ehtsham Muhammad Rotilio Marianna Cucchiella Federica Di Giovanni Gianni Schettini Domenico |
| author_sort | Ehtsham Muhammad |
| collection | DOAJ |
| description | Machine Learning (ML) models have been introduced in the past, and users have debated whether to tune the hyperparameters of the models. This study investigates the effects of tuning the hyperparameters of the ML models and summarizes the models that are most sensitive to hyperparameter tuning. This study leveraged the historic energy production data of two already operational PV plants. Four state-of-the-art ML models, namely Decision Trees (DT), Random Forest (RF), K-Nearest Neighbors (KNN), and Support Vector Regression (SVR) were investigated. All the ML models were trained with the same training features (meteorological estimates) obtained from the National Aeronautics and Space Administration’s (NASA) Power project, with the daily PV energy production selected as the target variable. Models were developed and executed with default and tuned hyperparameters using an 85-15% train-test split. The results revealed that all the models showed improved performance with the tuned hyperparameters. However, the DT and SVR models depicted significantly improved RMSE after tuning of the hyperparameters. The RMSE of DT improved from 111 kWh/d to 75 kWh/d for one plant and from 442 kWh/d to 270 kWh/d for the second plant after tuning the hyperparameters. Similarly, the RMSE of SVR improved from 59 kWh/d to 50 kWh/d in the first case, and in the second case, the improvement of RMSE from 536 kWh/d to 294 kWh/d was observed. The efficiency of the RF and KNN models also improved to some extent after tuning, but the RMSE closely agreed with the default hyperparameters in one case study, making the RF and KNN less prone to hyperparameter sensitivity. This study concluded with the finding that it is necessary to tune the hyperparameters of the DT and SVR models, specifically for energy forecasting. Moreover, the results of this study also highlight the significance of meteorological estimates from NASA’s Power project, as models successfully discerned the complex energy forecast patterns. The dataset is deemed suitable for energy forecasting for areas with sparse ground-based observatories and may serve as a baseline dataset for training the ML models. |
| format | Article |
| id | doaj-art-da3b3b839a834fdaaae07daf87e6dbba |
| institution | Kabale University |
| issn | 2267-1242 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | EDP Sciences |
| record_format | Article |
| series | E3S Web of Conferences |
| spelling | doaj-art-da3b3b839a834fdaaae07daf87e6dbba2025-02-05T10:51:05ZengEDP SciencesE3S Web of Conferences2267-12422025-01-016120100210.1051/e3sconf/202561201002e3sconf_aere2025_01002Investigating the effects of hyperparameter sensitivity on machine learning algorithms for PV forecastingEhtsham Muhammad0Rotilio Marianna1Cucchiella Federica2Di Giovanni Gianni3Schettini Domenico4Department of Civil, Construction-Architectural and Environmental Engineering, University ofDepartment of Civil, Construction-Architectural and Environmental Engineering, University ofDepartment of Industrial and Information Engineering and Economics, University ofDepartment of Civil, Construction-Architectural and Environmental Engineering, University ofDepartment of Industrial and Information Engineering and Economics, University ofMachine Learning (ML) models have been introduced in the past, and users have debated whether to tune the hyperparameters of the models. This study investigates the effects of tuning the hyperparameters of the ML models and summarizes the models that are most sensitive to hyperparameter tuning. This study leveraged the historic energy production data of two already operational PV plants. Four state-of-the-art ML models, namely Decision Trees (DT), Random Forest (RF), K-Nearest Neighbors (KNN), and Support Vector Regression (SVR) were investigated. All the ML models were trained with the same training features (meteorological estimates) obtained from the National Aeronautics and Space Administration’s (NASA) Power project, with the daily PV energy production selected as the target variable. Models were developed and executed with default and tuned hyperparameters using an 85-15% train-test split. The results revealed that all the models showed improved performance with the tuned hyperparameters. However, the DT and SVR models depicted significantly improved RMSE after tuning of the hyperparameters. The RMSE of DT improved from 111 kWh/d to 75 kWh/d for one plant and from 442 kWh/d to 270 kWh/d for the second plant after tuning the hyperparameters. Similarly, the RMSE of SVR improved from 59 kWh/d to 50 kWh/d in the first case, and in the second case, the improvement of RMSE from 536 kWh/d to 294 kWh/d was observed. The efficiency of the RF and KNN models also improved to some extent after tuning, but the RMSE closely agreed with the default hyperparameters in one case study, making the RF and KNN less prone to hyperparameter sensitivity. This study concluded with the finding that it is necessary to tune the hyperparameters of the DT and SVR models, specifically for energy forecasting. Moreover, the results of this study also highlight the significance of meteorological estimates from NASA’s Power project, as models successfully discerned the complex energy forecast patterns. The dataset is deemed suitable for energy forecasting for areas with sparse ground-based observatories and may serve as a baseline dataset for training the ML models.https://www.e3s-conferences.org/articles/e3sconf/pdf/2025/12/e3sconf_aere2025_01002.pdf |
| spellingShingle | Ehtsham Muhammad Rotilio Marianna Cucchiella Federica Di Giovanni Gianni Schettini Domenico Investigating the effects of hyperparameter sensitivity on machine learning algorithms for PV forecasting E3S Web of Conferences |
| title | Investigating the effects of hyperparameter sensitivity on machine learning algorithms for PV forecasting |
| title_full | Investigating the effects of hyperparameter sensitivity on machine learning algorithms for PV forecasting |
| title_fullStr | Investigating the effects of hyperparameter sensitivity on machine learning algorithms for PV forecasting |
| title_full_unstemmed | Investigating the effects of hyperparameter sensitivity on machine learning algorithms for PV forecasting |
| title_short | Investigating the effects of hyperparameter sensitivity on machine learning algorithms for PV forecasting |
| title_sort | investigating the effects of hyperparameter sensitivity on machine learning algorithms for pv forecasting |
| url | https://www.e3s-conferences.org/articles/e3sconf/pdf/2025/12/e3sconf_aere2025_01002.pdf |
| work_keys_str_mv | AT ehtshammuhammad investigatingtheeffectsofhyperparametersensitivityonmachinelearningalgorithmsforpvforecasting AT rotiliomarianna investigatingtheeffectsofhyperparametersensitivityonmachinelearningalgorithmsforpvforecasting AT cucchiellafederica investigatingtheeffectsofhyperparametersensitivityonmachinelearningalgorithmsforpvforecasting AT digiovannigianni investigatingtheeffectsofhyperparametersensitivityonmachinelearningalgorithmsforpvforecasting AT schettinidomenico investigatingtheeffectsofhyperparametersensitivityonmachinelearningalgorithmsforpvforecasting |