![](https://coding-bootcamps.com/wp-content/uploads/2024/01/84-1.jpg)
Note
If you are new to the blockchain technology, taking our Introduction to Blockchain Technology self-paced course is highly recommended. Also, for a comprehensive coverage of blockchain development in Ethereum or mastering Solidity programming, taking our below self paced courses is highly recommended:
Recap
In our previous article (A roadmap for Implementing Ethereum 2.0), we covered the roadmap for implementing version 2 of Ethereum.
In this article, we learn about how to work with decentralized data and content storages like IPFS or BigChainDB in Ethereum.
Working with decentralized data and content storage
Data and content management are two of the main capabilities in many of the real-world business applications, such as information portals, Wikipedia, and ecommerce and social media applications. There is no exception in the decentralized world. During the EVM discussion, we briefly looked at the EVM capability for storing data on Ethereum.
Although it is convenient, it is not generally intended to be used for data storage. It is very expensive too. There are a few options application developers can leverage to manage and access decentralized data and contents for decentralized applications, including Swarm (the Ethereum blockchain solution), IPFS and BigchainDB (a big data platform for blockchain). We will cover them in details in our next article series.
Swarm
Swarm provides a content distribution service for Ethereum and DApps. Here are some features of Swarm:
- It is a decentralized storage platform, a native base layer service of the Ethereum web 3 stack.
- It intends to be a decentralized store of Ethereum’s public record as an alternative to an Ethereum on-chain storage solution.
- It allows DApps to store and distribute code, data, and contents, without jamming all the information on the blockchain.
Imagine you are developing a blockchain-based medical record system, you want to keep track when the medical records are added, where the medical records are recorded, and who has accessed the medical records and for what purpose. All these are the immutable transaction records you want to maintain in the blockchain. But, the medical records themselves, including physician notes, medical diagnosis, and imaging, and so on, may not be suitable to be stored in the Ethereum blockchain. Swarm or IPFS are best suited for such use cases.
A typical Ethereum DApp application architecture for such use cases may look like the diagram here:
DApps can create, manage, and store data and content directly into a decentralized file system, like IPFS and Swarm, and access and retrieve the data and content using a Swarm hash. When DApps submit all transactions to the Ethereum network, the transactions can reference the Swarm resources with the referenced Swarm hash.
One-to-One Live Blockchain Classes
Coding Bootcamps school offers One-to-One Live Blockchain Classes for Beginners.
Internally, Swarm maintains a specific type of content addressed distributed hash table (DHT) across the decentralized nodes. File or content uploaded into the Swarm network is treated as the blobs, and chopped into different chunks. A Merkle tree is then created out of all those chunks, and is used to ensure the content integrity. Trunks are further distributed to participating nodes and stored into the DHT. When an access request is made, the content is served by the node(s) closest to the address of a chunk.
Swarm offers several APIs for accessing and managing the contents, including a
CLI (command-line interface) and JSON-RPC APIs. JavaScript packages are available through the erebos, swarm-js or swarmgw packages, which can be leveraged by most of the UI/JavaScript- based DApps.
IPFS
IPFS is similar to Swarm; it is a peer-to-peer distributed filesystem that was designed to store and share the content across a decentralized network. Both IPFS and Swarm offer the decentralized data and content storage with content addressable hash, generated directly from the content. Both are used to store any kind of content, which can be referenced from the transactions in the Blockchain network.
Behind the scenes, there are quite a few technical differences; mainly, in terms how each chop large datasets into chunks and store them in a distributed network. IPFS may be thought of as a single BitTorrent swarm, exchanging objects within one Git repository. Swarm may be seen as more integrated with the Ethereum blockchain, and has an incentivized system for content sharing. However, Filecoin can be an overlay on top of IPFS for providing a similar incentivized system.
The DApp application architecture in the Swarm section applies to IPFS too. In the same way, IPFS offers several APIs for accessing and managing the contents, including a CLI interface, JSON-RPC APIs, and an HTTP interface. JavaScript packages and Go library are available too, which can be leveraged by most of the UI/JavaScript or Go-based DApps.
BigchainDB
BigchainDB is a decentralized database combining both traditional database and data management capabilities and blockchain features. As a blockchain database, BigchainDB is complementary to other decentralized systems, such as decentralized file storage like IPFS or Swarm, and smart contract blockchains like Ethereum or EOS. It is another alternative for storing decentralized data and content. It can be used as the data storage for traditional applications, or can be leveraged as the decentralized data storage for decentralized blockchain platforms, like Ethereum. Although it can be used as a file repository, it is not recommended since it is best suited to structured or unstructured data.
Within the Ethereum community, there is a lot of interest in integrating BigchainDB with Ethereum smart contracts. Some EIPs and POCs (prototype of concept) were proposed to explore such integration options. One of the PoCs is to leverage the Oraclize service to retrieve data from BigchainDB within a smart contract. On successful retrieval of data, the smart contract evaluates and executes the logic and performs the requested operation. As the diagram shows, there are two ways a DApp can integrate with BigChainDB. One is to directly interact with BigchainDB as the decentralized data storage through HTTP GET and POST. The second option is to leverage Oraclize service in smart contracts to access external data from BigChainDB:
This process uses the following rules:
- BigchainDB offers several interfaces for connecting to BigchainDB servers and storing and retrieving data from the blockchain database, including a CLI interface and HTTP APIs.
- When storing data in the database, you will need to use HTTP POST to send the data to the database server. You use the HTTP GET interface to retrieve data from the database.
- BigchainDB also provides database drivers for developers to connect to the network servers from high-level programming languages, like Java, Python. and JavaScript/Node.js.
Next Article
In our next article (How Decentralized Messaging with Whisper Works in Ethereum), we discuss how decentralized messaging with Whisper works in Ethereum.
This article is written in collaboration with Brian Wu who is a leading author of “Learn Ethereum: Build your own decentralized applications with Ethereum and smart contracts” book. He has written 7 books on blockchain development.
Resources
Free Webinars on Blockchain
Here is the list of our free webinars that are highly recommended:
- Hyperledger Fabric for system admin versus developers
- How to harness blockchain for environmental and corporate sustainability
- Review of Initial Coin Offering, Security Token Offering and asset tokenization use cases and best practices
- Hyperledger Fabric Deployment on Cloud
- Hyperledger Fabric for entrepreneurship- 21 blockchain business use cases
Free Courses
Here is the list of our 10 free self-paced courses that are highly recommended:
- IT Career Roadmap Explained
- Web Design with Bootstrap
- User Experience Best Practices
- Intro to Search Engine Optimization
- Web Design with WordPress
- Introduction to Drupal CMS
- Intro to Joomla CMS
- Intro to Cybersecurity
- Introduction to Cloud Technology
- Recorded Live Webinars and Classes
Self-Paced Blockchain Courses
If you like to learn more about Hyperledger Fabric, Hyperledger Sawtooth, Ethereum or Corda, taking the following self-paced classes is highly recommended:
- Intro to Blockchain Technology
- Blockchain Management in Hyperledger for System Admins
- Hyperledger Fabric for Developers
- Intro to Blockchain Cybersecurity
- Learn Solidity Programming by Examples
- Introduction to Ethereum Blockchain Development
- Learn Blockchain Dev with Corda R3
- Intro to Hyperledger Sawtooth for System Admins
Live Blockchain Courses
If you want to master Hyperledger Fabric, Ethereum or Corda, taking the following live classes is highly recommended:
- Live and self-paced blockchain development with Ethereum
- Live and self-paced blockchain development with Hyperledger Fabric
- Live and self-paced blockchain development with Corda
- Immersive Blockchain Bootcamp with live and self-paced courses
- Live crash course for learning Ethereum with Solidity
- Live crash course for building DApps with Hyperledger Fabric
- Live crash course for building DApps with Corda
- Live full-stack blockchain development in Hyperledger and Ethereum
Articles and Tutorials on Blockchain Technology
If you like to learn more about blockchain technology and how it works, reading the following articles is highly recommended:
- History and Evolution of Blockchain Technology from Bitcoin
- Overview of Blockchain evolution and phases from Ethereum to Hyperledger
- Comprehensive overview and analysis of blockchain use cases in many industries
- Blockchain Crowdfunding Security Token or Initial Coin Offerings
- A beginner Guide to Blockchain Technology
- How Decentralized Peer-To-Peer Network Works
- How blocks are added to the blockchain
- How Public and Private Keys of Cryptography Work
- What Is A Cryptographic Hash Function
- How Digital Signature Works In Blockchain
- The role and types of consensus mechanism in blockchain
- How Proof-of-Work Consensus Works in Blockchain
- How Proof of Stake Consensus works in Blockchain
Articles and Tutorials on Ethereum and Solidity
If you like to learn more about blockchain development in Ethereum with Solidity, reading the following articles and tutorials is highly recommended:
- Review of Architecture and Components of Ethereum
- Comprehensive Blockchain Ethereum Developer Guide from Beginner to Advance Level
- How to Write Ethereum Smart Contracts with Solidity in 1 hour
- Review of Architecture and Components of Ethereum
- How Ethereum Manages Accounts
- How Ethereum Manages Transactions
- How Smart Contracts Work in Ethereum
- How Ether and Gas Work in Ethereum
- How Ethereum Virtual Machine works
- How address and wallet work in Ethereum
- How mining works in Ethereum
- List of Tools and Technologies in Ethereum Ecosystem
- Review of challenges in distributed systems
- Review of Cap Theorem in Distributed Systems
- Horizontal Scaling versus Vertical Scaling in Distributed Systems
- How to Scale up Ethereum Blockchain Applications
- Review of scaling solutions for Ethereum
- How to Manage Ethereum State Channel with Raiden
- How Plasma Chains Work in Ethereum
- How Sharding and Gasper work in Ethereum
- How Proof-of-Stack Consensus Works in Ethereum
- A roadmap for Implementing Ethereum 2.0
- How to work with Decentralized Data and Content Storage in Ethereum
- How Decentralized Messaging with Whisper Works in Ethereum
- Review of Infura for Ethereum Development
- Review of Infura Ethereum API
- How to Use Remix with Infura for Ethereum Development
- How Ethereum Client API Works
- How Ethereum IPFS Storage Works
- How to Install and Start Ethereum IPFS Storage
- How to Run Ethereum IPFS Storage
- How to Work with Ethereum Swarm Storage
- How to Install Ethereum Swarm Storage
- How to Handle Ethereum Messages with Whisper
- Review of Popular Ethereum Smart Contract Libraries
- Review of Private and Permissioned blockchain
- How to Set up a Local Private Ethereum Blockchain
- How to Run Geth on a Local Private Ethereum Blockchain
- How to Build a Local Private Ethereum Blockchain with Mining
- How to Run Geth on a Local Private Ethereum Blockchain with Mining
- How to Create an Account on a Local Private Ethereum Blockchain
- How to Use Ethereum Optional Flags with New Chains
- Review of Ethereum Options for Development and Testing
- Review of Ethereum Developer Chain Options
- Review of Ethereum API and Console Options
- Review of Ethereum Networking Options
- Review of Ethereum Transaction Pool Options
Articles and Tutorials on Hyperledger Family
If you like to learn more about blockchain development with Hyperledger, reading the following articles and tutorials is highly recommended:
- Introduction to Hyperledger Architecture, Projects, Tools and Libraries
- Complete Review of Hyperledger Fabric Architecture and Components
- Hyperledger Fabric for System Administers versus Developers
- How to use Prometheus and Grafana to monitor Hyperledger Fabric
- Blockchain Developer Guide- How to Install Hyperledger Fabric on AWS
- Blockchain Developer Guide- How to Install and work with Hyperledger Sawtooth
- Blockchain Developer Guide- How to Install Hyperledger Burrow on AWS
- Blockchain Developer Guide- How to Install Hyperledger Iroha on AWS
- Blockchain Developer Guide- How to Install Hyperledger Indy and Indy CLI on AWS
- Blockchain Developer Guide- How to Install Hyperledger Seth and Docker on AWS
- Blockchain Developer Guide- How to Configure Hyperledger Sawtooth Validator and REST API on AWS
- Blockchain Developer Guide- How to Build Transaction Processor as a Service and Python Egg for Hyperledger Sawtooth
- Blockchain Developer Guide- How to Deploy Ethereum Smart Contracts with Hyperledger Burrow
- Blockchain Developer Guide- How to Create Cryptocurrency Using Hyperledger Iroha CLI
- Blockchain Developer Guide- How to Explore Hyperledger Indy Command Line Interface
- Blockchain Developer Guide- Comprehensive Blockchain Hyperledger Developer Guide from Beginner to Advance Level
- Introduction to Hyperledger Sawtooth Blockchain Development
Articles and Tutorials on R3 Corda
If you like to learn more about blockchain development on Corda , reading the following articles and tutorials is highly recommended:
Articles and Tutorials on Other Blockchain Platforms
If you like to learn more about blockchain development in other platforms, reading the following articles and tutorials is highly recommended: