Note
If you are new to the blockchain technology, taking our Introduction to Blockchain Technology self-paced course is highly recommended. Also, for a comprehensive coverage of blockchain development in Ethereum or mastering Solidity programming, taking our below self paced courses is highly recommended:
Recap
In our previous series, we covered the following 12 articles:
- Review of Private and Permissioned blockchain
- How to Set up a Local Private Ethereum Blockchain
- How to Run Geth on a Local Private Ethereum Blockchain
- How to Build a Local Private Ethereum Blockchain with Mining
- How to Run Geth on a Local Private Ethereum Blockchain with Mining
- How to Create an Account on a Local Private Ethereum Blockchain
- How to Use Ethereum Optional Flags with New Chains
- Review of Ethereum Options for Development and Testing
- Review of Ethereum Developer Chain Options
- Review of Ethereum API and Console Options
- Review of Ethereum Networking Options
- Review of Ethereum Transaction Pool Options
This article series provides the concluding articles where we will briefly touch on a few fascinating areas, including cross-blockchain interoperability, technology fusion, and the convergence between blockchain, Artificial Intelligence/Machine Learning, and Internet of Things.
Facing the challenges of Ethereum and blockchain
There are quite a few challenges that Ethereum and blockchain are facing. We will go through them in the following sections.
Consensus protocol and scalability
As we discussed in our earlier series, Deep Research on Ethereum, there are several active work and research streams in the Ethereum community racing to solve Ethereum’s scalability issues. Obviously, the Ethereum ecosystem has evolved considerably since it was invented 4-5 years ago. However, for the mainstream adoption of Ethereum and blockchain technology in general, scalability is the most pressing issue. The solution to the scalability issue holds the key to making Ethereum sink or swim.
Let’s recap on the current state of scalability challenges and analyze various approaches and activities in improving scalability in Ethereum:
- Scalability issues in Ethereum: In Ethereum and blockchain, scalability occurred because every transaction needs to be processed and verified by all the nodes in the decentralized network. For the network to agree on the true state of the blockchain network, there are consensus protocols involved which allow the network of untrusted nodes to reach consensus. In general, blockchain compromises scalability and throughput in favor of decentralization and the security of the network. In this article, we’ll take a step back and discuss the common challenges that are faced in scaling Ethereum.In the abstract sense, when a transaction happens, it is sent to the network and relayed to all the network nodes. It will be packaged into a block, which will be added to the blockchain by the mining node, and then verified by all the nodes. The whole process involves totally uncoordinated, but largely interdependent, activities to make a transaction recorded and secured in the network. It involves the communication of transactions and blocks, proof of the transaction’s validity and verification of it, and an agreement on the finality of the transactions. Think of Ethereum as a heart valve. At its current capacity, transactions are pumped in and out at a slow and steady pace. The network is operating as expected. But as transaction volume picks up, or any part of the network becomes clogged, the system becomes inadequate for running critical business and finance operations.Simply adding network nodes won’t help at all. Solutions such as increasing block size or packaging more in a block have their limits. In our previous articles ( Horizontal Scaling versus Vertical Scaling in Distributed Systems & How to Scale up Ethereum Blockchain Applications & Review of scaling solutions for Ethereum), we discussed the block size increase solution. There are some proposals to reexamine the block structure and Merkle tree, with an intention to be able to effectively package more transactions in a block. It may be worth reading. Such solutions may help in the short term, but they alone won’t fundamentally address these issues. There were solutions for increasing the block creation’s frequency that didn’t fly, simply due to the fact they may cause long finality and instability of the blockchain. That leaves most of the focus on proof, verification, and finality.
- Sharding and Layer 1 solutions: As we discussed in our other article, the Ethereum community has settled into a multi-pronged strategy in addressing scalability through the Ethereum 2.0 roadmap. Shifting to a Proof-of-Stake(PoS) consensus with Casper CBC is the ultimate goal. In the short term and as a tactical step, Ethereum is moving to a hybrid PoS and Proof-of-Work(PoW) model which seems to be a practical and prudent approach.Ethereum sharding is the audacious attempt to address the fundamental issues with blockchain scalability. It intends to find a scalable solution without compromising network security and decentralization, two tenets of blockchain technology. Sharding is not a new concept and has been proven effective in managing large distributed database systems. The success with sharding in the traditional world comes from two factors: one is that the solution itself typically involves a central mediator or intermediary to make sharding and aggregation coordination, while the other is that software vendors have a central authority or governance structure in making sharding solution decisions, and deciding where to compromise and what subset of the issues to address and where to market.The complexity and risks in implementing Ethereum sharding increase immensely. Not only will the blockchain and world state be sharded, but the entire decentralized peer-to-peer network will be grouped into different shards as well. The complexity lies in enforcing data and network security while being able to coordinate those transaction activities and aggregate data and transactions across blockchain data and network segments. For that purpose, Ethereum Casper introduces the beacon chain layer as the decentralized alternative to the central mediator in the traditional sharding implementation. A mix of PoW and PoS consensus protocol, as well as the finality solution of Casper FFG, will certainly add to the complexity of sharding implementation. How it works is still largely a work in progress.
- Off-chain computation with Zero-Knowledge Succinct Non-Interactive Argument of Knowledge and multiparty computation: Ethereum is envisioned as the world computer, but people quickly realized that, in the Ethereum architecture’s current form, there are huge expenses associated with that vision and the consequence in scalability and throughput this has caused. One of those is computations. Another one is on-chain storage. We talked a little about zk- SNARKs in our earlier articles. This is another promising strategy that offloads expensive encryption computation out of the Ethereum Virtual Machine(EVM) to third-party verifiers, without compromising privacy and security, to improve scalability and throughput in Ethereum. Vitalik thinks that, by using zk-SNARKs to mass-validate transactions, we can actually scale asset transfer transactions on Ethereum to about 500 transactions per second. We have seen such success in Zcash. It is one of the areas the Ethereum community is closely watching. More research and prototypes have yet to be seen before we can see a clear roadmap or a future implementation.
- Multiparty computation (MPC) is another area that may have the potential for addressing privacy and scalability issues. In an MPC model, a set of participants, P1, P2, …, Pn, are assigned a subset of private data, D1, D2, …, Dn, respectively. The participants want to compute the value of a public function on that private data while keeping their own inputs secret. While pairing with blockchain, the subset of blockchain data (mainly the reference to the public blockchain), will be stored privately in some kind of distributed hash table(DTH) within each MPC node.
One-to-One Live Blockchain Classes
Coding Bootcamps school offers One-to-One Live Blockchain Classes for Beginners.
The following steps explain the process flow for some MPC blockchains such as Platon:
- Deploy the smart contract to the blockchain.
- Invoke the smart contract function by providing computation algorithm parameters.
- The computation task will distribute the contract to an off-chain MPC network.
- The MPC nodes will verify the computation and generate the required proof.
- The on-chain nodes will broadcast the result and proof.
- The block producers verify the proof.
- The verified result will be returned to the user through a smart contract.
- The miner will get coin as an economic reward.
The computation is offloaded and done securely through the coordination of all the MPC nodes, without anyone knowing anything from other MPC nodes. The idea behind this is that, by offloading computations out of the main chain, it will stay leaner and more scalable.
- Layer 2 solutions: There are discussions in the Ethereum community and Ethereum 2.0 roadmap about reexamining Ethereum’s blockchain data structure, as well as the storage model. Hopefully, a solution will be created to make the Ethereum client stateless and leverage it more for enforcing security and integrity. A good compromise would be to offload those to Layer 2 or off-chain solutions.
We will continue to see Layer 2 scaling solutions take shape in Ethereum. We talked about state channel solutions with Raiden and side-chain solutions with Plasma in our earlier articles and discussed how the Layer 2 solution would work. As the Ethereum ecosystem evolves, and more and more transactions need to be processed, it definitely makes much more sense to explore the Layer 2 solutions. Their purpose is to offload expensive computations and transaction processing out of Mainnet and leverage Ethereum as the root chain for security guarantees and enforcement. State channels, as the perfect solution for micropayments, will continue to flourish in the payment space. Plasma, as the side-chains for many different verticals, will probably be the main avenue for Ethereum to expand beyond the finance, banking, and payment industry.
Next Article
In our next article ( UI/UX and Design Thinking for Ethereum Blockchain Development), we discuss how to use UI/UX and design thinking best practices while building Ethereum blockchain applications.
This article is written in collaboration with Brian Wu who is a leading author of “Learn Ethereum: Build your own decentralized applications with Ethereum and smart contracts” book. He has written 7 books on blockchain development.
Resources
Free Webinars on Blockchain
Here is the list of our free webinars that are highly recommended:
- Hyperledger Fabric for system admin versus developers
- How to harness blockchain for environmental and corporate sustainability
- Review of Initial Coin Offering, Security Token Offering and asset tokenization use cases and best practices
- Hyperledger Fabric Deployment on Cloud
- Hyperledger Fabric for entrepreneurship- 21 blockchain business use cases
Free Courses
Here is the list of our 10 free self-paced courses that are highly recommended:
- IT Career Roadmap Explained
- Web Design with Bootstrap
- User Experience Best Practices
- Intro to Search Engine Optimization
- Web Design with WordPress
- Introduction to Drupal CMS
- Intro to Joomla CMS
- Intro to Cybersecurity
- Introduction to Cloud Technology
- Recorded Live Webinars and Classes
Self-Paced Blockchain Courses
If you like to learn more about Hyperledger Fabric, Hyperledger Sawtooth, Ethereum or Corda, taking the following self-paced classes is highly recommended:
- Intro to Blockchain Technology
- Blockchain Management in Hyperledger for System Admins
- Hyperledger Fabric for Developers
- Intro to Blockchain Cybersecurity
- Learn Solidity Programming by Examples
- Introduction to Ethereum Blockchain Development
- Learn Blockchain Dev with Corda R3
- Intro to Hyperledger Sawtooth for System Admins
Live Blockchain Courses
If you want to master Hyperledger Fabric, Ethereum or Corda, taking the following live classes is highly recommended:
- Live and self-paced blockchain development with Ethereum
- Live and self-paced blockchain development with Hyperledger Fabric
- Live and self-paced blockchain development with Corda
- Immersive Blockchain Bootcamp with live and self-paced courses
- Live crash course for learning Ethereum with Solidity
- Live crash course for building DApps with Hyperledger Fabric
- Live crash course for building DApps with Corda
- Live full-stack blockchain development in Hyperledger and Ethereum
Articles and Tutorials on Blockchain Technology
If you like to learn more about blockchain technology and how it works, reading the following articles is highly recommended:
- History and Evolution of Blockchain Technology from Bitcoin
- Overview of Blockchain evolution and phases from Ethereum to Hyperledger
- Comprehensive overview and analysis of blockchain use cases in many industries
- Blockchain Crowdfunding Security Token or Initial Coin Offerings
- A beginner Guide to Blockchain Technology
- How Decentralized Peer-To-Peer Network Works
- How blocks are added to the blockchain
- How Public and Private Keys of Cryptography Work
- What Is A Cryptographic Hash Function
- How Digital Signature Works In Blockchain
- The role and types of consensus mechanism in blockchain
- How Proof-of-Work Consensus Works in Blockchain
- How Proof of Stake Consensus works in Blockchain
Articles and Tutorials on Ethereum and Solidity
If you like to learn more about blockchain development in Ethereum with Solidity, reading the following articles and tutorials is highly recommended:
- Review of Architecture and Components of Ethereum
- Comprehensive Blockchain Ethereum Developer Guide from Beginner to Advance Level
- How to Write Ethereum Smart Contracts with Solidity in 1 hour
- Review of Architecture and Components of Ethereum
- How Ethereum Manages Accounts
- How Ethereum Manages Transactions
- How Smart Contracts Work in Ethereum
- How Ether and Gas Work in Ethereum
- How Ethereum Virtual Machine works
- How address and wallet work in Ethereum
- How mining works in Ethereum
- List of Tools and Technologies in Ethereum Ecosystem
- Review of challenges in distributed systems
- Review of Cap Theorem in Distributed Systems
- Horizontal Scaling versus Vertical Scaling in Distributed Systems
- How to Scale up Ethereum Blockchain Applications
- Review of scaling solutions for Ethereum
- How to Manage Ethereum State Channel with Raiden
- How Plasma Chains Work in Ethereum
- How Sharding and Gasper work in Ethereum
- How Proof-of-Stack Consensus Works in Ethereum
- A roadmap for Implementing Ethereum 2.0
- How to work with Decentralized Data and Content Storage in Ethereum
- How Decentralized Messaging with Whisper Works in Ethereum
- Review of Infura for Ethereum Development
- Review of Infura Ethereum API
- How to Use Remix with Infura for Ethereum Development
- How Ethereum Client API Works
- How Ethereum IPFS Storage Works
- How to Install and Start Ethereum IPFS Storage
- How to Run Ethereum IPFS Storage
- How to Work with Ethereum Swarm Storage
- How to Install Ethereum Swarm Storage
- How to Handle Ethereum Messages with Whisper
- Review of Popular Ethereum Smart Contract Libraries
- Review of Private and Permissioned blockchain
- How to Set up a Local Private Ethereum Blockchain
- How to Run Geth on a Local Private Ethereum Blockchain
- How to Build a Local Private Ethereum Blockchain with Mining
- How to Run Geth on a Local Private Ethereum Blockchain with Mining
- How to Create an Account on a Local Private Ethereum Blockchain
- How to Use Ethereum Optional Flags with New Chains
- Review of Ethereum Options for Development and Testing
- Review of Ethereum Developer Chain Options
- Review of Ethereum API and Console Options
- Review of Ethereum Networking Options
- Review of Ethereum Transaction Pool Options
Articles and Tutorials on Hyperledger Family
If you like to learn more about blockchain development with Hyperledger, reading the following articles and tutorials is highly recommended:
- Introduction to Hyperledger Architecture, Projects, Tools and Libraries
- Complete Review of Hyperledger Fabric Architecture and Components
- Hyperledger Fabric for System Administers versus Developers
- How to use Prometheus and Grafana to monitor Hyperledger Fabric
- Blockchain Developer Guide- How to Install Hyperledger Fabric on AWS
- Blockchain Developer Guide- How to Install and work with Hyperledger Sawtooth
- Blockchain Developer Guide- How to Install Hyperledger Burrow on AWS
- Blockchain Developer Guide- How to Install Hyperledger Iroha on AWS
- Blockchain Developer Guide- How to Install Hyperledger Indy and Indy CLI on AWS
- Blockchain Developer Guide- How to Install Hyperledger Seth and Docker on AWS
- Blockchain Developer Guide- How to Configure Hyperledger Sawtooth Validator and REST API on AWS
- Blockchain Developer Guide- How to Build Transaction Processor as a Service and Python Egg for Hyperledger Sawtooth
- Blockchain Developer Guide- How to Deploy Ethereum Smart Contracts with Hyperledger Burrow
- Blockchain Developer Guide- How to Create Cryptocurrency Using Hyperledger Iroha CLI
- Blockchain Developer Guide- How to Explore Hyperledger Indy Command Line Interface
- Blockchain Developer Guide- Comprehensive Blockchain Hyperledger Developer Guide from Beginner to Advance Level
- Introduction to Hyperledger Sawtooth Blockchain Development
Articles and Tutorials on R3 Corda
If you like to learn more about blockchain development on Corda , reading the following articles and tutorials is highly recommended:
Articles and Tutorials on Other Blockchain Platforms
If you like to learn more about blockchain development in other platforms, reading the following articles and tutorials is highly recommended: