TSB-Forecast: A Short-Term Load Forecasting Model in Smart Cities for Integrating Time Series Embeddings and Large Language Models
In smart cities, energy management systems are essential for efficient resource utilization, enhanced operational efficiency, and sustainability promotion. This work presents a novel load forecasting model, TSB-Forecast (Time Series BERT), a hybrid machine learning model aiming to improve short-term...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11121830/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849735776855654400 |
|---|---|
| author | Mohamed Mahmoud Hasan Neamat El-Tazi Ramadan Moawad Amany H. B. Eissa |
| author_facet | Mohamed Mahmoud Hasan Neamat El-Tazi Ramadan Moawad Amany H. B. Eissa |
| author_sort | Mohamed Mahmoud Hasan |
| collection | DOAJ |
| description | In smart cities, energy management systems are essential for efficient resource utilization, enhanced operational efficiency, and sustainability promotion. This work presents a novel load forecasting model, TSB-Forecast (Time Series BERT), a hybrid machine learning model aiming to improve short-term electrical demand forecasting by integrating structured and unstructured data. The model uses Sentence Bidirectional Encoder Representations from Transformers (SBERT) to extract semantic characteristics from textual news and Time to Vector (Time2Vec) to capture temporal patterns, acquiring cyclical behavior and context-sensitive impacts. TSB-Forecast shows significant improvements reducing MAE by 38.7%, RMSE by 18.2%, and SMAPE by 50.6% compared to several baselines, including the official ENTSO-E forecast, Extra Trees Regressor (ETR), a hybrid model with basic text processing (M6), and a deep learning model combining GRU and CNN (CNN-GRU). TSB-Forecast demonstrates a stronger ability to combine time-related and contextual information, which helps reduce errors. Our contributions in this paper are fourfold and can be summarized as follows: The model initially integrates multi-source inputs, encompassing load demand, BBC News, meteorological data, and holiday information. Secondly, it utilizes Time2Vec and SBERT for improved temporal and semantic feature encoding. Subsequently, Extreme Gradient Boosting (XGBoost) and Extra Trees Regressor (ETR) collectively enhance the robustness of a stacked ensemble. Finally, a thorough empirical analysis showing consistent outperformance of the proposed model over all the forecasting measures against baseline models. |
| format | Article |
| id | doaj-art-e038efe8754e416d90b5f6d5857f2b8b |
| institution | DOAJ |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-e038efe8754e416d90b5f6d5857f2b8b2025-08-20T03:07:27ZengIEEEIEEE Access2169-35362025-01-011314169414171610.1109/ACCESS.2025.359742111121830TSB-Forecast: A Short-Term Load Forecasting Model in Smart Cities for Integrating Time Series Embeddings and Large Language ModelsMohamed Mahmoud Hasan0https://orcid.org/0000-0002-6350-4762Neamat El-Tazi1https://orcid.org/0000-0003-0690-4273Ramadan Moawad2https://orcid.org/0000-0003-2349-8203Amany H. B. Eissa3https://orcid.org/0000-0001-8663-2604Department of Information Systems, Faculty of Computers and Information Technology, Future University in Egypt, Cairo, EgyptDepartment of Information Systems, Faculty of Computers and Artificial Intelligence, Cairo University, Cairo, EgyptDepartment of Computer Science, Faculty of Computers and Information Technology, Future University in Egypt, Cairo, EgyptDepartment of Information Systems, Faculty of Computers and Artificial Intelligence, Cairo University, Cairo, EgyptIn smart cities, energy management systems are essential for efficient resource utilization, enhanced operational efficiency, and sustainability promotion. This work presents a novel load forecasting model, TSB-Forecast (Time Series BERT), a hybrid machine learning model aiming to improve short-term electrical demand forecasting by integrating structured and unstructured data. The model uses Sentence Bidirectional Encoder Representations from Transformers (SBERT) to extract semantic characteristics from textual news and Time to Vector (Time2Vec) to capture temporal patterns, acquiring cyclical behavior and context-sensitive impacts. TSB-Forecast shows significant improvements reducing MAE by 38.7%, RMSE by 18.2%, and SMAPE by 50.6% compared to several baselines, including the official ENTSO-E forecast, Extra Trees Regressor (ETR), a hybrid model with basic text processing (M6), and a deep learning model combining GRU and CNN (CNN-GRU). TSB-Forecast demonstrates a stronger ability to combine time-related and contextual information, which helps reduce errors. Our contributions in this paper are fourfold and can be summarized as follows: The model initially integrates multi-source inputs, encompassing load demand, BBC News, meteorological data, and holiday information. Secondly, it utilizes Time2Vec and SBERT for improved temporal and semantic feature encoding. Subsequently, Extreme Gradient Boosting (XGBoost) and Extra Trees Regressor (ETR) collectively enhance the robustness of a stacked ensemble. Finally, a thorough empirical analysis showing consistent outperformance of the proposed model over all the forecasting measures against baseline models.https://ieeexplore.ieee.org/document/11121830/Load forecastingtime series embeddingssemantic embeddings |
| spellingShingle | Mohamed Mahmoud Hasan Neamat El-Tazi Ramadan Moawad Amany H. B. Eissa TSB-Forecast: A Short-Term Load Forecasting Model in Smart Cities for Integrating Time Series Embeddings and Large Language Models IEEE Access Load forecasting time series embeddings semantic embeddings |
| title | TSB-Forecast: A Short-Term Load Forecasting Model in Smart Cities for Integrating Time Series Embeddings and Large Language Models |
| title_full | TSB-Forecast: A Short-Term Load Forecasting Model in Smart Cities for Integrating Time Series Embeddings and Large Language Models |
| title_fullStr | TSB-Forecast: A Short-Term Load Forecasting Model in Smart Cities for Integrating Time Series Embeddings and Large Language Models |
| title_full_unstemmed | TSB-Forecast: A Short-Term Load Forecasting Model in Smart Cities for Integrating Time Series Embeddings and Large Language Models |
| title_short | TSB-Forecast: A Short-Term Load Forecasting Model in Smart Cities for Integrating Time Series Embeddings and Large Language Models |
| title_sort | tsb forecast a short term load forecasting model in smart cities for integrating time series embeddings and large language models |
| topic | Load forecasting time series embeddings semantic embeddings |
| url | https://ieeexplore.ieee.org/document/11121830/ |
| work_keys_str_mv | AT mohamedmahmoudhasan tsbforecastashorttermloadforecastingmodelinsmartcitiesforintegratingtimeseriesembeddingsandlargelanguagemodels AT neamateltazi tsbforecastashorttermloadforecastingmodelinsmartcitiesforintegratingtimeseriesembeddingsandlargelanguagemodels AT ramadanmoawad tsbforecastashorttermloadforecastingmodelinsmartcitiesforintegratingtimeseriesembeddingsandlargelanguagemodels AT amanyhbeissa tsbforecastashorttermloadforecastingmodelinsmartcitiesforintegratingtimeseriesembeddingsandlargelanguagemodels |