A Scalable Framework for Sensor Data Ingestion and Real-Time Processing in Cloud Manufacturing
Cloud Manufacturing enables the integration of geographically distributed manufacturing resources through advanced Cloud Computing and IoT technologies. This paradigm promotes the development of scalable and adaptable production systems. However, existing frameworks face challenges related to scalab...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Algorithms |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-4893/18/1/22 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832589412925440000 |
|---|---|
| author | Massimo Pacella Antonio Papa Gabriele Papadia Emiliano Fedeli |
| author_facet | Massimo Pacella Antonio Papa Gabriele Papadia Emiliano Fedeli |
| author_sort | Massimo Pacella |
| collection | DOAJ |
| description | Cloud Manufacturing enables the integration of geographically distributed manufacturing resources through advanced Cloud Computing and IoT technologies. This paradigm promotes the development of scalable and adaptable production systems. However, existing frameworks face challenges related to scalability, resource orchestration, and data security, particularly in rapidly evolving decentralized manufacturing settings. This study presents a novel nine-layer architecture designed specifically to address these issues. Central to this framework is the use of Apache Kafka for robust, high-throughput data ingestion, and Apache Spark Streaming to enhance real-time data processing. This framework is underpinned by a microservice-based architecture that ensures a high scalability and reduced latency. Experimental validation using sensor data from the UCI Machine Learning Repository demonstrated substantial improvements in processing efficiency and throughput compared with conventional frameworks. Key components, such as RabbitMQ, contribute to low-latency performance, whereas Kafka ensures data durability and supports real-time application. Additionally, the in-memory data processing of Spark Streaming enables rapid and dynamic data analysis, yielding actionable insights. The experimental results highlight the potential of the framework to enhance operational efficiency, resource utilization, and data security, offering a resilient solution suited to the demands of modern industrial applications. This study underscores the contribution of the framework to advancing Cloud Manufacturing by providing detailed insights into its performance, scalability, and applicability to contemporary manufacturing ecosystems. |
| format | Article |
| id | doaj-art-877cb0a9bafb4dc396df25414177ad76 |
| institution | Kabale University |
| issn | 1999-4893 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Algorithms |
| spelling | doaj-art-877cb0a9bafb4dc396df25414177ad762025-01-24T13:17:30ZengMDPI AGAlgorithms1999-48932025-01-011812210.3390/a18010022A Scalable Framework for Sensor Data Ingestion and Real-Time Processing in Cloud ManufacturingMassimo Pacella0Antonio Papa1Gabriele Papadia2Emiliano Fedeli3Department of Engineering for Innovation, University of Salento, 73100 Lecce, ItalyDepartment of Science and Information Technology, Pegaso University, 80121 Napoli, ItalyDepartment of Engineering for Innovation, University of Salento, 73100 Lecce, ItalyLaser Romae S.r.l., 00144 Roma, ItalyCloud Manufacturing enables the integration of geographically distributed manufacturing resources through advanced Cloud Computing and IoT technologies. This paradigm promotes the development of scalable and adaptable production systems. However, existing frameworks face challenges related to scalability, resource orchestration, and data security, particularly in rapidly evolving decentralized manufacturing settings. This study presents a novel nine-layer architecture designed specifically to address these issues. Central to this framework is the use of Apache Kafka for robust, high-throughput data ingestion, and Apache Spark Streaming to enhance real-time data processing. This framework is underpinned by a microservice-based architecture that ensures a high scalability and reduced latency. Experimental validation using sensor data from the UCI Machine Learning Repository demonstrated substantial improvements in processing efficiency and throughput compared with conventional frameworks. Key components, such as RabbitMQ, contribute to low-latency performance, whereas Kafka ensures data durability and supports real-time application. Additionally, the in-memory data processing of Spark Streaming enables rapid and dynamic data analysis, yielding actionable insights. The experimental results highlight the potential of the framework to enhance operational efficiency, resource utilization, and data security, offering a resilient solution suited to the demands of modern industrial applications. This study underscores the contribution of the framework to advancing Cloud Manufacturing by providing detailed insights into its performance, scalability, and applicability to contemporary manufacturing ecosystems.https://www.mdpi.com/1999-4893/18/1/22Cloud Manufacturinginnovative frameworklayered architecturesystem scalabilityresource managementhigh-demand workloads |
| spellingShingle | Massimo Pacella Antonio Papa Gabriele Papadia Emiliano Fedeli A Scalable Framework for Sensor Data Ingestion and Real-Time Processing in Cloud Manufacturing Algorithms Cloud Manufacturing innovative framework layered architecture system scalability resource management high-demand workloads |
| title | A Scalable Framework for Sensor Data Ingestion and Real-Time Processing in Cloud Manufacturing |
| title_full | A Scalable Framework for Sensor Data Ingestion and Real-Time Processing in Cloud Manufacturing |
| title_fullStr | A Scalable Framework for Sensor Data Ingestion and Real-Time Processing in Cloud Manufacturing |
| title_full_unstemmed | A Scalable Framework for Sensor Data Ingestion and Real-Time Processing in Cloud Manufacturing |
| title_short | A Scalable Framework for Sensor Data Ingestion and Real-Time Processing in Cloud Manufacturing |
| title_sort | scalable framework for sensor data ingestion and real time processing in cloud manufacturing |
| topic | Cloud Manufacturing innovative framework layered architecture system scalability resource management high-demand workloads |
| url | https://www.mdpi.com/1999-4893/18/1/22 |
| work_keys_str_mv | AT massimopacella ascalableframeworkforsensordataingestionandrealtimeprocessingincloudmanufacturing AT antoniopapa ascalableframeworkforsensordataingestionandrealtimeprocessingincloudmanufacturing AT gabrielepapadia ascalableframeworkforsensordataingestionandrealtimeprocessingincloudmanufacturing AT emilianofedeli ascalableframeworkforsensordataingestionandrealtimeprocessingincloudmanufacturing AT massimopacella scalableframeworkforsensordataingestionandrealtimeprocessingincloudmanufacturing AT antoniopapa scalableframeworkforsensordataingestionandrealtimeprocessingincloudmanufacturing AT gabrielepapadia scalableframeworkforsensordataingestionandrealtimeprocessingincloudmanufacturing AT emilianofedeli scalableframeworkforsensordataingestionandrealtimeprocessingincloudmanufacturing |