Black-Box for Blockchain Parameters Adjustment

This paper introduces a function for blockchain performance evaluation as a black-box. The function runs the Solana blockchain test network with the only differences between the main network in a configuration file and the physical network to operate in. The black-box takes setup parameters as input, launches blockchain in a cloud, emulates artificial users’ activity, and gives two outputs–transactions per second (tps) and drop rate. By default, the setup has six most important integer parameters and a network with three computers in the cloud, while one can vary eighty-nine parameters, the number of computers in the network and use local computers via black-box configuration files. The applied problem is to maximize the tps under a zero drop rate constraint. The black-box, like real blockchains, uses network communication, so reproducibility is an essential part of the design. We also provide an optimization baseline, showing the non-trivial results’ reachability.

data storage-transactions-into ordered batches-blocks-and 23 achieve consensus about new blocks. Besides classic correct-24 ness requirements, blockchains need specific ones [6], [7], 25 such as high performance regarding transactions per second 26 (tps), fast transaction confirmation, and chain quality. 27 The fast transaction conformation is a measure of latency 28 between a user broadcasting a transaction on the Internet and 29 the transaction inclusion in an agreed block. If users gener- 30 The associate editor coordinating the review of this manuscript and approving it for publication was Mehdi Sookhak . ate transactions independent of the block generation events 31 moments, the latency can not be less than half of the average 32 block generation time. Furthermore, if the mempool-the set 33 of the new correct transactions to be added to the blockchain-34 is small, the ongoing block will include the new transaction, 35 and the resulting latency will be close to half of the block 36 generation time. Blockchain needs to have a more significant 37 throughput than the flow of new transactions to avoid the 38 infinite growth of the mempool. The throughput regarding 39 tps is a standard measure of blockchain performance. 40 The throughput and latency compete. On the one hand, 41 one can have blocks without user transactions or one trans-42 action each, then one can generate blocks according to the 43 pre-agreed deterministic rule, and the average time between 44 blocks can be negligible. Nevertheless, the throughput will be 45 zero for empty blocks and small for one-transaction blocks. 46 On the other hand, one can allow unlimited blocks and 47 include all the transactions over a long period into a single 48 block but generate blocks once per one hundred years. In this 49 case, the overhead for consensus will be small, and one can 50 achieve the maximum throughput. However, one will wait 51 fifty years for transaction processing, which is infeasible for 52 VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ We currently have networks without access to change 109 parameters for research for a fixed blockchain architecture. 110 Nevertheless, the source code of the blockchain is publicly 111 available. So we can launch a test system in our environ-112 ment and vary parameters as we wish. Both network and 113 protocol parameters define the blockchain operation regime. 114 Some of the network parameters are observable, for example, 115 the current pool of unconfirmed transactions; some of the 116 network parameters are not observable (latent), for example, 117 blockchain network graph, connection latency, and band-118 width [27], [28]. Unobservable parameters play an essential 119 role in the performance; for example, average propagation 120 delay affects transaction latency, and it is helpful to estimate 121 them with a particular accuracy [29]. Latent parameters result 122 in the impossibility of emulating the network with the test 123 network but only simulating.

124
This paper is a continuation of the research [30], where we 125 presented a machine learning view on blockchain parameters 126 adjustment, designed a Solana blockchain-based testbed, and 127 performed a sensitivity analysis of the available parameters. 128 This paper presents a function for blockchain performance 129 evaluation in the form of an interactive black-box. The func-130 tion runs the Solana blockchain test network with the only 131 differences between the network in a configuration file and 132 the physical network to operate in. The main objective is to 133 maximize the throughput, while more problems, including 134 the results propagation to the network, are of interest too. The 135 rest of the paper is organized as follows. Section 2 views the 136 Solana blockchain as a black-box for parameter adjustment. 137 Section 3 introduces the developed black-box and details its 138 reproducibility. The optimization with a limited black-box 139 computation budget shows the reachability of non-trivial 140 results in Section 4. Finally, Section 5 concludes the paper.

142
Solana is a high-performance permissionless blockchain [18], 143 [31]. Its network consists of clusters-sets of validators work-144 ing together to serve client transactions and maintain the 145 ledger's integrity. Clusters may coexist. When two clusters 146 share a common genesis block, they attempt to converge. For 147 example, one set of clusters with a common genesis block is 148 a Solana network, whose token is in the top ten cryptocurren-149 cies by market capitalization as of July 2022 [9]. Otherwise, 150 each cluster ignores the existence of the others with different 151 genesis blocks. One sets a genesis configuration to create 152 a cluster and runs a validator. Additional validators then 153 register with any registered member of the cluster.

154
Clients send transactions to any validator's Transaction 155 Processing Unit (TPU) port. A validator forwards the transac-156 tion to the designated leader. If the validator is in the lead role, 157 the node bundles incoming transactions, timestamps them, 158 creating an entry, and pushes them onto the cluster's data 159 plane. Once on the data plane, the transactions are validated 160 by validator nodes, effectively appending them to the ledger. 161 Solana defines confirmation as the duration of time from the 162 leader timestamps a new transaction to the recognition of a 163    Note that tps counts only successful transactions.

184
The high-load stationary [32] operating regimes are of 185 interest. We achieve them by launching a cluster on our  The uncontrollable parameters x = x 0 are constant for a 207 given testnet. We treat latent parameters ξ as independent 208 random variables for different runs. The testnet takes around 209 ten minutes to evaluate a random function y = f (θ ) = 210 F(x 0 , θ, ξ ). Under computational time constraint and lack of 211 an analytic model for the function f , a black-box optimization 212 problem arises: maximize the mean value of f (θ) over θ with 213 a maximum N max calls of f . and Z 6 denotes the set of all six-dimensional integers. 238 The output is two-dimensional: tps and drop rate. The 239 applied problem is to maximize tps under the zero drop rate 240 constraint.

241
The black-box is a computational job as follows 242 (see Figure 1) 243 1) Allocate computers within a single data centre. 244 By default, we use four t2.2xlarge virtual 245 The reason for a performance peak of around 30 seconds is 285 unknown. Such effect is not present on a local testnet but 286 only in the cloud. So the cloud resource allocation mechanism 287 may be the clue. Delays ≥ 70 seconds show stable stationary 288 results. We set the delay of 80 seconds as a default timeout 289 in a black-box for network convergence, but one can set his 290 value as an optional black-box parameter.

C. SIGNAL-TO-NOISE RATIO
292 The function f (θ ) = F(x 0 , θ, ξ ) is random. Signal-to-noise 293 ratio (SNR) as a measure that compares the level of a desired 294 signal to the level of background noise [34] 295     The standard deviation depends on the point, so the random 448 noise is not stationary over the domain (1). Optimal points 449 by Random, Metaheuristic and Surrogate are far 450 from each other, which shows either a locality for extrema 451 or a big impact of the noise. 453 Blockchain architectures keep various parameters constant 454 during their life without a justification for default values. 455 We have the mainnet without access to change parameters 456 for research and publicly available source code for a fixed 457 system. So one can launch a testnet in one's environment 458 and vary parameters as one wishes. Uncontrollable latent 459 parameters introduce randomness and make it impossible to 460 emulate the mainnet with the testnet but only simulate it. 461 The testnet can reveal the blockchain architecture behaviour, 462 check the operation regime sensitivity to the parameters cho-463 sen, justify the default parameters, and propagate the results 464 to the mainnet.

465
This paper introduces a Solana blockchain testnet as a 466 black-box function. By default, the black-box takes six 467 the most important parameters as input and returns two 468 blockchain performance parameters, namely, tps and drop 469 rate. The experiments show that the black-box noise is 470 significantly smaller than the output variation over the input 471 domain. The black-box is still not reproducible but has an 472 acceptable signal-to-noise ratio (SNR