If you are new to the blockchain technology, taking our Introduction to Blockchain Technology self-paced course is highly recommended. Also, for a comprehensive coverage of blockchain development in Ethereum or mastering Solidity programming, taking our below self paced courses is highly recommended:
In our previous article (How Plasma Chains Work in Ethereum), we covered how Plasma Chains work in Ethereum.
In this article, we learn how Sharding and Gasper work in Ethereum.
As we mentioned earlier, sharding has been implemented in many distributed database systems. Essentially, sharding is a particular method for horizontally partitioning a large dataset within a database. More specifically, the database is broken into little pieces called shards. When aggregated together, they form the original database. In the following diagram, one large dataset can be sliced horizontally into two or more partitions, and each partition may be stored in separate database instances:
In a decentralized blockchain network, the network consists of a series of nodes connected in a peer-to-peer format, with no central authority. As is the case with current blockchain system, each node stores all states of the network and processes all of the transactions.
In order to match transaction scalability and throughput from Mastercard or Visa, Ethereum will have to process large volumes of transactions much faster. One logical way we discussed earlier is to increase the block size, but this may not be able to fundamentally address the scalability issues, since every node still needs to validate and verify a large block of more transactions. They will need more computing resources to handle the increased load. That is where sharding comes to help.
Not only the large volume of transactions can be sharded, but the entire peer-to-peer network of nodes can be grouped into smaller groups, also called shards. This greatly improves the parallel execution in the network, and scalability.
Imagine at any period, in order to process a large volume of transactions, you would need to have a really large block to be able to hold all the transactions submitted in this period. Instead of creating a larger physical block, what if you slice them into different smaller blocks, as in the following diagram, and have smaller blocks stored in the subset of network nodes? Similar to database sharding, in order to find all transactions and verify any particular transactions, you have to aggregate the transactions from all shards for that period. When you aggregate all transactions, you will have a virtual large block:
It looks like a simple concept, but complexity arises when dealing with maintaining the integrity and security of the Ethereum world state and transaction ledger across a subset of network partitions. Like any other blockchains, Ethereum maintains transactions and their ordering in the chain of blocks through a consensus mechanism. Furthermore, Ethereum also manages the account as a state object in the work state. A state object makes things much more complicated.
Let us take a look at how the transactions and data work in Ethereum sharding. As depicted in the following diagram, let us say the network is split into multiple groups and each group is an Ethereum shard. Transactions pertaining to each shard will be collected and added to the shard chain. A group of validators in the collator pool, stake their coins and become the validators in the PoS system. Let's take a look at the following diagram:
Shard chain is similar to blockchain in concept. The miner's role in Ethereum is split into three separate roles: collation proposer, collator, and executor. A collation is the same as a block in the blockchain. As the following diagram shows, at any time period, a collation proposer collects all transactions, and adds them to a proposed collation. Data blobs from transactions are packaged together into the collation body, a Merkle tree of the data blobs, and other shard identifier information is packaged as the collation header.
The collation proposer proposes the proposed collation to the collator, which is randomly selected from the validator pool. The collator's role in this process is to validate the proposed collations and pass the collation header to smart contracts through the executor on the main chain. The collation headers from all shards will be added to blocks on the main blockchain:
The smart contract is also called a validator manager contract (VMC), as depicted in the preceding diagram. It is a shard-specific smart contract, deployed on the main chain. The VMC main role is to maintain the entire sharding systems and act as the connection between the main chain and the shard chain it links to. Its responsibilities include the following:
Cross-shard transactions, as discussed earlier, are one of the challenges that the sharding system faces. When a transaction happens across different shards, it is typically implemented through the cross-shard communications of transactions and receipts.
For example, suppose Alice from shard 1 needs to pay 100ETH to Bob on shard 2. The single transaction will be broken into two separate transactions: one is a debit transaction to draw 100ETH from Alice's account on shard 1, and another one is the credit transaction of 100ETH made to Bob on shard 2. The following are the steps involved:
Sharding, along with the PoS consensus mechanism, has the potential to drastically improve Ethereum's scalability and increase the transaction throughput in Ethereum. There are too many moving parts in combining both layer 1 scalability solutions and creating a solid implementation. It has been handled in multiple phases, we will go over the current Ethereum 2.0 roadmap in later articles to discuss which different components should be implemented and when.
In our next article (How Proof-of-Stack Consensus Works in Ethereum), we discuss how the Proof-of-Stack consensus is used in Ethereum.
This article is written in collaboration with Brian Wu who is a leading author of “Learn Ethereum: Build your own decentralized applications with Ethereum and smart contracts” book. He has written 7 books on blockchain development.
Here is the list of our free webinars that are highly recommended:
Here is the list of our 10 free self-paced courses that are highly recommended:
If you like to learn more about Hyperledger Fabric, Hyperledger Sawtooth, Ethereum or Corda, taking the following self-paced classes is highly recommended:
If you want to master Hyperledger Fabric, Ethereum or Corda, taking the following live classes is highly recommended:
If you like to learn more about blockchain technology and how it works, reading the following articles is highly recommended:
If you like to learn more about blockchain development in Ethereum with Solidity, reading the following articles and tutorials is highly recommended:
If you like to learn more about blockchain development with Hyperledger, reading the following articles and tutorials is highly recommended:
If you like to learn more about blockchain development on Corda , reading the following articles and tutorials is highly recommended:
If you like to learn more about blockchain development in other platforms, reading the following articles and tutorials is highly recommended:
We offer private custom tutoring classes both online and in DC, MD and VA for almost all of our courses or bootcamps. Give us a call or email us to discuss your needs.
$50 Limited OfferREGISTER NOW