BlockLoader: A Comprehensive Evaluation Framework for Blockchain Performance under Various Workload Patterns

Hyperledger Fabric is one of the most popular permissioned blockchain platforms widely adopted in enterprise blockchain solutions. To optimize and fully utilize the platform, it is desired to conduct a thorough performance analysis of Hyperledger Fabric. Although numerous studies have analyzed the performance of Hyperledger Fabric, three significant limitations still exist. First, existing blockchain performance evaluation frameworks rely on fixed workload rates, which fail to accurately reflect the performance of blockchain systems in real-world application scenarios. Second, the impact of extending the breadth and depth of endorsement policies on the performance of blockchain systems has yet to be adequately studied. Finally, the impact of node crashes and recoveries on blockchain system performance has yet to be comprehensively investigated. To address these limitations, we propose a framework called BlockLoader, which offers seven different distributions of load rates, including linear, single-peak, and multi-peak patterns. Next, we employ the BlockLoader framework to analyze the impact of endorsement policy breadth and depth on blockchain performance, both qualitatively and quantitatively. Additionally, we investigate the impact of dynamic node changes on performance. The experimental results demonstrate that different endorsement policies exert distinct effects on performance regarding breadth and depth scalability. In the horizontal expansion of endorsement policies, the OR endorsement policy demonstrates stable performance, fluctuating around 88 TPS, indicating that adding organizations and nodes has minimal impact. In contrast, the AND endorsement policy exhibits a declining trend in performance as the number of organizations and nodes increases, with an average decrease of 10 TPS for each additional organization. Moreover, the dynamic behaviour of nodes exerts varying impacts across these endorsement policies. Specifically, under the AND endorsement policy, dynamic changes in nodes significantly affect system performance. The TPS of the AND endorsement policy shows a notable decline, dropping from 79.6 at 100 s to 41.96 at 500 s, reflecting a reduction of approximately 47% over time. Under the OR endorsement policy, the system performance remains almost unaffected.