The core idea of ​​sharding technology is "divide and conquer". To give an inappropriate example, the sharding scheme is like opening multiple toll channels on a congested high-speed toll station to ensure traffic speed and reduce road traffic pressure. Sharding technology still faces many problems, including: each shard stores too much data, and nodes need to synchronize a large amount of data when switching shards; sharding means that the number of nodes participating in a certain transaction verification and consensus is reduced, and it is easy to be controlled by malicious attackers ; Interaction problems caused by cross-chip transactions, etc.

Editor | Lu Xiaoming

At present, sharding technology (Sharding) is an important direction of the public chain expansion plan, and is considered to be a longer-term and theoretically feasible expansion plan. Capacity expansion usually refers to the ability of the system to handle high-frequency traffic. The main problem to be solved by the expansion scheme is to realize the large-scale application of blockchain technology by improving the performance processing capability of the blockchain system.

This article on Odaily will introduce that MultiVAC is also a public chain project that uses sharding technology to expand capacity. They also proposed a concept of "elastic computing network" that is different from the cloud computing field, that is, DApp developers based on the public chain can choose different dimensions such as scalability, security, and decentralization according to their own business needs. , the underlying network will use this to assign contract transactions to different shards. Developers also need to deploy contracts according to their own business.

The core idea of ​​sharding technology is "divide and conquer". To give an inappropriate example, the sharding scheme is like opening multiple toll channels on a congested high-speed toll station to ensure traffic speed and reduce road traffic pressure. Sharding technology still faces many problems, including: each shard stores too much data, and nodes need to synchronize a large amount of data when switching shards; sharding means that the number of nodes participating in a certain transaction verification and consensus is reduced, and it is easy to be controlled by malicious attackers ; Interaction problems caused by cross-chip transactions, etc.

image description

Split node role: reduce storage costs

Split node role: reduce storage costs

The first feature of the MultiVAC is,The key point of this design is that the miner node can directly use the summary information of the transaction + the transaction + the proof provided by the storage node to verify the transaction. Only storing transaction summary information reduces the data volume of miner nodes and lowers the entry threshold for miners. Ordinary PCs can also join.

The reason why the blockchain needs fragmentation is that the transaction volume is too large to be carried by a single-chain system. At the same time, with the skyrocketing data volume, ordinary laptops may not be enough to store the entire network ledger.

In order to reduce stored data and ensure decentralization, MultiVAC introduces the role of storage nodes and divides nodes into three types: storage nodes, miner nodes and light nodes, none of which need to have a full network ledger.

The miner nodes in the shard first elect the block producing node, verify and package the transaction (the block producer needs to call the Merkle Proof to the storage node to prove that the input is not used), broadcast in the shard, and then send it to the storage node for synchronization.

The function of the miner node is to store summary information (block header), and is responsible for verifying transactions and having control over the data. The storage node stores all the ledgers in the shard, which is somewhat similar to the role of a super node. It has a large storage capacity and has certain access thresholds, but the difference is that it only stores transactions and assists in transaction verification, without any control over the data, ensuring other Decentralized features. MultiVAC CTO Ying Xiang said that storage nodes can be understood as network node basic service providers, similar to the role of network providers (China Mobile, China Unicom, and China Telecom).

The key point of this design is that the miner node can directly use the summary information of the transaction + the transaction + the proof provided by the storage node to verify the transaction. Only storing transaction summary information reduces the data volume of miner nodes and lowers the entry threshold for miners. Ordinary PCs can also join.

According to MultiVAC,The corresponding storage nodes in each shard of MultiVAC are fixed, but the miner nodes are not fixed. With the help of VRF transformation, the storage nodes have no control over the data. Since the miner nodes only store summary information and the total amount is very small, there is no need to worry about the amount of data migrated by miners. Otherwise, miners still need to synchronize a large amount of ledger data when changing shards, or need to have the entire network ledger. Ying Xiang believes that this also causes the switching cycle of most sharding schemes to be long (meaning unsafe), and only a part of nodes can be switched at a time.

Security and decentralization go hand in hand. There is a degradation of security in sharding, and security requires complete decentralization. Miner nodes and storage nodes designed by MultiVAC also play a role here.

Suppose a shard has 100 nodes. To attack the shard, you only need to know what the 100 nodes are, and the security risk is very high. Therefore, most sharding schemes need to periodically change the nodes in the shard. The same is true for MultiVAC, every few minutes, the shard where the miner is located will switch.

The corresponding storage nodes in each shard of MultiVAC are fixed, but the miner nodes are not fixed. With the help of VRF transformation, the storage nodes have no control over the data. Since the miner nodes only store summary information and the total amount is very small, there is no need to worry about the amount of data migrated by miners. Otherwise, miners still need to synchronize a large amount of ledger data when changing shards, or need to have the entire network ledger. Ying Xiang believes that this also causes the switching cycle of most sharding schemes to be long (meaning unsafe), and only a part of nodes can be switched at a time.

The advantage of this is that the cost of cross-shard transactions can be "fixed", and will not increase with the increase of cross-shard transactions, and the increased storage cost is very small. "Assuming that the size of a block header for a shard is about 100 bytes (byte), if there are 64 shards, it is about 6KB."

Fixed transmission volume to ensure cross-shard transactions independent of shards

The problem that sharding technology has to face is the realization of cross-shard transactions. If there is no cross-shard transaction, there is no interaction between each shard, and it is no different from a single chain.

There is a view that transactions in the blockchain are atomic, and the operations involved must be guaranteed to achieve final consistency, that is, either all of them are completed or none of them start. Therefore, the operations between the shards have to be coordinated to ensure their correct operation. according to"Wang Jiaping: Five Big Lies of the Technology Known as "Public Chain Fragmentation""One article: "At present, the common processing method is to use the concept of thread synchronization to lock the state involved in the transaction, prevent other non-related transactions from interfering with these states, and release it until all operations of the transaction are completed. But the problem with this processing method is that , in the locked state, other executions of some shards are blocked, and other tasks of the shards cannot be carried out normally, and in this case, as the number of shards increases, the number of cross-shard transactions increases, and the resulting blockage becomes more obvious."

Ying Xiang said that MultiVAC's cross-chip transaction adopts a different idea. Each shard is guaranteed to be independent.

Each shard of MultiVAC will regularly synchronize the block headers of other shards. Each block in a shard has a space to record the latest status of other shards, and process cross-shard transactions accordingly.

After shard 1 initiates a transaction at a certain moment, the transaction is confirmed at block height h1; (at this time, the height of shard 2 is h2), and this transaction is reflected on shard 2, which needs to be in shard 2 A certain height (usually the next block, such as h2+1) reads the height of h1 from shard 1 before adding money to shard 2. It’s a bit like here, Bank A remits a sum of money to Bank B’s account. Bank A shows that the money has been deducted, but Bank B may not receive it immediately. During this period, network congestion across shards may cause transaction delays, but it will not Affect the normal progress of other transactions in the shard. MultiVAC uses the final consistency method to ensure that shard 2 can obtain all transfers from shard 1 in order.

The advantage of this is that the cost of cross-shard transactions can be "fixed", and will not increase with the increase of cross-shard transactions, and the increased storage cost is very small. "Assuming that the size of a block header for a shard is about 100 bytes (byte), if there are 64 shards, it is about 6KB."

I am Odaily reporter Suixin (WeChat ID wsuixin12) and editor Lu Xiaoming (lohiuming). Please note your name, unit, title and reason when adding friends.

Focus on sharing economic applications

What Odaily wonders is that the MultiVAC solution saves storage costs, but increases a lot of transmission costs. Will it affect the implementation of the solution? In this regard, Ying Xiang responded: "Yes, it increases the amount of data to be transmitted by 70%, but for example, Java is 10 times slower than C++, but many people use Java because the development cost is much lower. The same is true here."

He further explained the logic: storage nodes provide proofs so that miner nodes can verify whether transactions and block generation are legal without having ledgers and only having very simplified summary information. The price of doing this is that there is an extra part of data transmission (proof information). 70% is the amount of uncompressed data. In fact, after we compress it, it is about 20-25%. It can be compressed to a smaller size in the future.

But conversely, every transaction in the traditional blockchain is broadcast once across the entire network. Once the business volume increases, a single node will be overwhelmed. On the one hand, the sharding mechanism allows transactions to only be broadcast within the shard. On the other hand, for the data in the shard, MultiVAC sends unconfirmed transactions to the storage nodes, and the storage nodes can broadcast multiple transactions at a time, and the miners collect them into their own memory pools. Therefore, the number of transaction broadcasts has been greatly reduced, but the amount of broadcast data has increased by 20-25%. Generally speaking, the drag on the network load is not serious.

MultiVAC said that it has completed the development of the underlying transaction sharding solution. The TPS of a single shard is about 500; the contract sharding is still under development, and the application is expected to be launched in the second half of 2019. It pays more attention to the application practice of the sharing economy.

From the perspective of future development, sharding technology still faces many problems. The current mainstream sharding technology is divided into three levels: network sharding, transaction sharding, and state sharding, and its technical difficulty also increases in turn. There are still many problems to be solved in terms of specific implementation, such as consensus, PoS, network delay, etc.

In terms of consensus mechanism, Ying Xiang believes that there is no perfect consensus mechanism at present, and MultiVAC uses binary Byzantine + group signature.

As for the incentive mechanism, MultiVAC said that miner nodes are mainly mining revenue, and the incentive mechanism of storage nodes considers a competition (bidding nature) mechanism. Ideally, there will be a balance, similar to the gas fee of Ethereum.

The team currently has about 25 people, including 17 technical and 7 operational. The MultiVAC technical team has PhDs in computer science from Harvard, Stanford, Tsinghua University and Nanyang Technological University, as well as senior engineering experts from Google, Facebook and Meituan Dianping. CEO Lu Heng used to be the technical director of Meituan and the co-founder and CTO of Xinrenxinshi; CTO Ying Xiang is an associate professor of Tianjin University, a Ph.D. in computer science from Nanyang Technological University, and an expert in the field of parallel algorithms and general computing. CMO Wang Chen used to be the product manager of Meituan, the marketing operation director of Xinrenxinshi, and the co-founder of Juli Xinfang (formerly Kanjia.com).

According to reports, the team raised $15 million in mid-2018. According to the official website, investment institutions come from South Korea, the United States, Europe and other places, including IDG, Hongtai Capital,Jiuhe Venture Capitaland other investment institutions andNGC、JRR Crypto, Hashed, Arrington XRP and other digital asset investment funds.

I am Odaily reporter Suixin (WeChat ID wsuixin12) and editor Lu Xiaoming (lohiuming). Please note your name, unit, title and reason when adding friends.