BlockLoader: A Comprehensive Evaluation Framework for Blockchain Performance Under Various Workload Patterns
Hyperledger Fabric is one of the most popular permissioned blockchain platforms widely adopted in enterprise blockchain solutions. To optimize and fully utilize the platform, it is desired to conduct a thorough performance analysis of Hyperledger Fabric. Although numerous studies have analyzed the p...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-10-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/12/21/3403 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850197199429828608 |
|---|---|
| author | Gang Wang Yanfeng Zhang Chenhao Ying Qinnan Zhang Zhiyuan Peng Xiaohua Li Ge Yu |
| author_facet | Gang Wang Yanfeng Zhang Chenhao Ying Qinnan Zhang Zhiyuan Peng Xiaohua Li Ge Yu |
| author_sort | Gang Wang |
| collection | DOAJ |
| description | Hyperledger Fabric is one of the most popular permissioned blockchain platforms widely adopted in enterprise blockchain solutions. To optimize and fully utilize the platform, it is desired to conduct a thorough performance analysis of Hyperledger Fabric. Although numerous studies have analyzed the performance of Hyperledger Fabric, three significant limitations still exist. First, existing blockchain performance evaluation frameworks rely on fixed workload rates, which fail to accurately reflect the performance of blockchain systems in real-world application scenarios. Second, the impact of extending the breadth and depth of endorsement policies on the performance of blockchain systems has yet to be adequately studied. Finally, the impact of node crashes and recoveries on blockchain system performance has yet to be comprehensively investigated. To address these limitations, we propose a framework called BlockLoader, which offers seven different distributions of load rates, including linear, single-peak, and multi-peak patterns. Next, we employ the BlockLoader framework to analyze the impact of endorsement policy breadth and depth on blockchain performance, both qualitatively and quantitatively. Additionally, we investigate the impact of dynamic node changes on performance. The experimental results demonstrate that different endorsement policies exert distinct effects on performance regarding breadth and depth scalability. In the horizontal expansion of endorsement policies, the OR endorsement policy demonstrates stable performance, fluctuating around 88 TPS, indicating that adding organizations and nodes has minimal impact. In contrast, the AND endorsement policy exhibits a declining trend in performance as the number of organizations and nodes increases, with an average decrease of 10 TPS for each additional organization. Moreover, the dynamic behaviour of nodes exerts varying impacts across these endorsement policies. Specifically, under the AND endorsement policy, dynamic changes in nodes significantly affect system performance. The TPS of the AND endorsement policy shows a notable decline, dropping from 79.6 at 100 s to 41.96 at 500 s, reflecting a reduction of approximately 47% over time. Under the OR endorsement policy, the system performance remains almost unaffected. |
| format | Article |
| id | doaj-art-4d00791da4d347fd8e3ef10366d807b2 |
| institution | OA Journals |
| issn | 2227-7390 |
| language | English |
| publishDate | 2024-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj-art-4d00791da4d347fd8e3ef10366d807b22025-08-20T02:13:14ZengMDPI AGMathematics2227-73902024-10-011221340310.3390/math12213403BlockLoader: A Comprehensive Evaluation Framework for Blockchain Performance Under Various Workload PatternsGang Wang0Yanfeng Zhang1Chenhao Ying2Qinnan Zhang3Zhiyuan Peng4Xiaohua Li5Ge Yu6School of Computer Science and Engineering, Northeastern University, Shenyang 110167, ChinaSchool of Computer Science and Engineering, Northeastern University, Shenyang 110167, ChinaDepartment of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, ChinaBeijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University, Beijing 100191, ChinaDepartment of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, ChinaSchool of Computer Science and Engineering, Northeastern University, Shenyang 110167, ChinaSchool of Computer Science and Engineering, Northeastern University, Shenyang 110167, ChinaHyperledger Fabric is one of the most popular permissioned blockchain platforms widely adopted in enterprise blockchain solutions. To optimize and fully utilize the platform, it is desired to conduct a thorough performance analysis of Hyperledger Fabric. Although numerous studies have analyzed the performance of Hyperledger Fabric, three significant limitations still exist. First, existing blockchain performance evaluation frameworks rely on fixed workload rates, which fail to accurately reflect the performance of blockchain systems in real-world application scenarios. Second, the impact of extending the breadth and depth of endorsement policies on the performance of blockchain systems has yet to be adequately studied. Finally, the impact of node crashes and recoveries on blockchain system performance has yet to be comprehensively investigated. To address these limitations, we propose a framework called BlockLoader, which offers seven different distributions of load rates, including linear, single-peak, and multi-peak patterns. Next, we employ the BlockLoader framework to analyze the impact of endorsement policy breadth and depth on blockchain performance, both qualitatively and quantitatively. Additionally, we investigate the impact of dynamic node changes on performance. The experimental results demonstrate that different endorsement policies exert distinct effects on performance regarding breadth and depth scalability. In the horizontal expansion of endorsement policies, the OR endorsement policy demonstrates stable performance, fluctuating around 88 TPS, indicating that adding organizations and nodes has minimal impact. In contrast, the AND endorsement policy exhibits a declining trend in performance as the number of organizations and nodes increases, with an average decrease of 10 TPS for each additional organization. Moreover, the dynamic behaviour of nodes exerts varying impacts across these endorsement policies. Specifically, under the AND endorsement policy, dynamic changes in nodes significantly affect system performance. The TPS of the AND endorsement policy shows a notable decline, dropping from 79.6 at 100 s to 41.96 at 500 s, reflecting a reduction of approximately 47% over time. Under the OR endorsement policy, the system performance remains almost unaffected.https://www.mdpi.com/2227-7390/12/21/3403hyperledger fabricBlockLoaderendorsement policynode dynamic changes |
| spellingShingle | Gang Wang Yanfeng Zhang Chenhao Ying Qinnan Zhang Zhiyuan Peng Xiaohua Li Ge Yu BlockLoader: A Comprehensive Evaluation Framework for Blockchain Performance Under Various Workload Patterns Mathematics hyperledger fabric BlockLoader endorsement policy node dynamic changes |
| title | BlockLoader: A Comprehensive Evaluation Framework for Blockchain Performance Under Various Workload Patterns |
| title_full | BlockLoader: A Comprehensive Evaluation Framework for Blockchain Performance Under Various Workload Patterns |
| title_fullStr | BlockLoader: A Comprehensive Evaluation Framework for Blockchain Performance Under Various Workload Patterns |
| title_full_unstemmed | BlockLoader: A Comprehensive Evaluation Framework for Blockchain Performance Under Various Workload Patterns |
| title_short | BlockLoader: A Comprehensive Evaluation Framework for Blockchain Performance Under Various Workload Patterns |
| title_sort | blockloader a comprehensive evaluation framework for blockchain performance under various workload patterns |
| topic | hyperledger fabric BlockLoader endorsement policy node dynamic changes |
| url | https://www.mdpi.com/2227-7390/12/21/3403 |
| work_keys_str_mv | AT gangwang blockloaderacomprehensiveevaluationframeworkforblockchainperformanceundervariousworkloadpatterns AT yanfengzhang blockloaderacomprehensiveevaluationframeworkforblockchainperformanceundervariousworkloadpatterns AT chenhaoying blockloaderacomprehensiveevaluationframeworkforblockchainperformanceundervariousworkloadpatterns AT qinnanzhang blockloaderacomprehensiveevaluationframeworkforblockchainperformanceundervariousworkloadpatterns AT zhiyuanpeng blockloaderacomprehensiveevaluationframeworkforblockchainperformanceundervariousworkloadpatterns AT xiaohuali blockloaderacomprehensiveevaluationframeworkforblockchainperformanceundervariousworkloadpatterns AT geyu blockloaderacomprehensiveevaluationframeworkforblockchainperformanceundervariousworkloadpatterns |