Decentralized Storage Report: The Foundation of WEB3.0 Development

Decentralized storage is important because it addresses the various pain points of centralized cloud storage in Web 2.0 and better meets the needs of the development of the big data era. It can store unstructured edge data at a lower cost and with higher efficiency, empowering technologies such as IoT and autonomous driving.

Our society is currently in an unprecedented era of information explosion. Computers, smart devices, televisions, home security systems, wearable devices, cars, and even robots are constantly generating and using data. With the development of AI and the Internet of Things (IoT), a vast array of devices will produce a massive amount of data.

The future will be a digital era where data becomes the main factor of production, with an exponential increase in data volume posing higher demands on the current data storage system, including services for data storage, management, and retrieval.

Traditional centralized cloud storage is a storage solution that puts storage resources on the cloud for people to access. The business model of internet cloud storage as a service has a long history. Amazon Web Services (AWS) was launched by Amazon in 2006, renting out their servers and storage space to users, which reduced the cost for developers to create and manage server infrastructure.

Currently, the internet cloud service market is already very large, with a market size of 200 billion US dollars in 2021. Companies such as Amazon, Microsoft, Google from abroad, and Alibaba from China represent the leading enterprises in centralized cloud storage.

Overview of Internet cloud storage business

Amazon Web Services (AWS) has consistently maintained a leading position in the industry, with its revenue reaching 62 billion US dollars in 2021, accounting for about 33% of the cloud service market share. Microsoft ranks second, with astonishing growth in recent years, increasing its market share from 13% in 2017 to 21% in 2021.

As for Amazon Web Services, there are four core departments: Cloud Computing, Cloud Storage, Networking, and Software. The revenue growth of each department is as follows:

The data in the chart shows that the fastest-growing sector is software services, with revenue reaching nearly 7 billion US dollars in the third quarter of 2021. The growth curves and revenue of cloud computing, cloud storage, and network services are relatively close, at around 3 billion US dollars, which has doubled compared to the revenue level in 2018.

In summary, in the era of big data, there is still a very broad market growth space for cloud storage.

As time progresses, centralized cloud storage services continue to improve and prices continue to drop. However, due to the centralization of data in centralized storage, the amount of data involved is greater, making data more susceptible to attacks and leaks. There are certain risks to the security, privacy, and sustainability of centralized storage data, causing the industry to gradually hit a bottleneck.

The security of data includes two layers of meaning: the first layer is “ensuring that data privacy is not leaked,” and the second layer is “ensuring that data integrity is not lost.”

Under the current centralized storage model, users upload all sensitive data, not only losing control over their own data but also transferring the risk of data leakage to the cloud storage operator.

If privacy information such as data loss, damage, leakage, or misuse occurs, it may lead to significant losses for individuals, businesses, and even the entire society, and tarnish the reputation of cloud storage operators.

At the same time, cloud storage itself is nothing more than a business model. Centralized cloud storage operators may run away or stop services due to various market or self-management issues, and users cannot constrain or claim against the service provider’s actions.

This leads users to tend to store data with larger, more credible service providers, resulting in an increasing degree of data centralization among leading companies. This also means that if data is lost, there will be a large-scale loss.

In terms of sustainability, the current centralized storage model has certain service limitations. With the popularization of unmanned driving and the Internet of Things (IoT), a large number of smart devices will connect to the network and generate exponentially increasing real-time data.

In this context, centralized data storage obviously cannot meet the needs of network storage. Future data storage systems must not only store, share, and read data but also transmit and analyze data efficiently and accurately, posing a great challenge to the centralized data storage structure.

Pain points of centralized storage

In summary, the vulnerability of centralized data storage models is somewhat detrimental to the development of the big data era. To fundamentally address issues such as frequent data breaches, service limitations, and operator shutdowns in traditional centralized storage, the concept of decentralized/distributed storage has gradually emerged.

Decentralized Storage Network (DSN) is a storage business model that uses blockchain technology as its underlying technology, storing files or sets of files in slices on the storage space provided by suppliers through distributed storage.

The importance of decentralized storage lies in its ability to solve the various pain points of centralized cloud storage in Web 2.0, and it is more in line with the needs of the development of the big data era. It can store unstructured edge data at a lower cost and with higher efficiency, empowering technologies such as IoT and autonomous driving. At the same time, decentralized storage is also the cornerstone of the development of Web 3.0.

3.1 Decentralized Storage Addresses the Pain Points of Centralized Cloud Storage in Web2

The analysis above states that traditional internet uses centralized storage where all data is stored on application platform servers, which currently faces many issues such as user data security, ownership, privacy protection, and sustainability.

The advantage of decentralized storage lies in the fact that data can be replicated across multiple locations and accessed from various places. This reduces the security issues of hackers attacking through a single node, effectively carries out data rights confirmation and privacy protection, and allows users to have full control over their own data. Such security and privacy are not available in centralized networks.

In addition, projects led by Arweave focus on the issue of permanent data storage, contributing to the long-term stability of important data storage. It can be said that the challenges faced by centralized storage are precisely the opportunities that decentralized storage welcomes.

3.2 Distributed Storage for Unstructured Edge Data

Structured data refers to the content stored in traditional relational databases, while unstructured data refers to data stored in the form of ordinary files, such as images, audio, video, and documents. According to an IDC report, 75% of the future data increment will be unstructured edge data.

Unlike centralized cloud storage services, distributed storage can migrate data storage from remote cloud servers to edge storage devices or edge data centers that are closer to the data for local storage, resulting in lower network communication overhead, interaction latency, and bandwidth costs, and higher adaptability and scalability, making it suitable for handling large amounts of data at high speeds.

Why should decentralized storage be worth paying attention to?

3.3 Decentralized Storage Cost Advantages

Decentralized data storage, especially for unstructured data, has significant cost advantages. This is because distributed storage changes the production relationship through nodes and token rewards, allowing end-users to use storage, indexing, and other services at a very low price.

For example, storing NFTs using the IPFS network costs almost nothing. In contrast, traditional internet cloud storage services, such as Baidu Cloud, actually have very high centralized operation costs. Therefore, from a cost perspective, decentralized storage has a clear competitive advantage.

3.4 Decentralized Storage is the Cornerstone of Web3.0

Decentralized storage is one of the earliest and most focused infrastructures in the Web3 field. The earliest decentralized storage solutions, Storj and the IPFS protocol, were launched in 2017.

Without distributed storage, there would be no decentralized network, and there would be no Web3.0. Projects that only have decentralized services but lack underlying decentralized storage support cannot be strictly considered a Web3 project. Therefore, decentralized storage is the cornerstone that empowers the long-term development of Web3, which is of great significance.

In summary, decentralized storage not only solves the problems of traditional centralized storage but also serves the data storage needs of traditional industries in the era of big data and empowers the emerging Web3.0. As an important infrastructure in the encryption field, the business model of the decentralized storage track is clear and the market size is showing a significant growth trend, which is profitable for miners and investors.

With the development of the Web3 ecosystem and the emphasis on privacy and user data ownership, the decentralized storage field will bring significant returns.

The development of decentralized storage has gone through different stages, each addressing different issues:

Storj and Sia implemented P2P supply and demand matching through smart contracts, which is suitable for large-scale data storage. However, their essence is the docking of storage resources, lacking an effective content addressing method, which is not conducive to file sharing, such as movies, audio, etc.

IPFS has achieved file addressing, which is a significant innovation in the underlying protocols of the Internet. Projects like Filecoin and Crust have built the IPFS storage network through incentive layers, thereby providing stable decentralized storage services.

Arweave achieves permanent file storage through an innovative consensus mechanism.

The Ceramic database storage protocol, which was launched in 2021, can meet the needs of high-frequency data. Stratos has introduced a Web3 decentralized infrastructure that integrates blockchain, storage, and computing.

However, even after five years of development, decentralized storage is still in its early stages. Currently, the total effective storage capacity of the Filecoin network is 59.6PB, and the total storage capacity of Arweave has reached 76.3TB. According to statistics from Web3 Index, Arweave’s storage costs over the past 90 days were $185,000, while Storj’s were $55,000.

It is evident that decentralized storage is currently far behind centralized storage in terms of both storage capacity and performance. However, given the various advantages of decentralized storage, there is still significant growth potential in this field, and it is expected to unlock more possibilities in the future.

5.1 What is Filecoin?

Before understanding Filecoin, we need to understand IPFS.

IPFS (InterPlanetary File System) is a decentralized network protocol that went live in 2015, primarily targeting the Hypertext Transfer Protocol (HTTP) on the internet, aiming to complement and even replace it.

The IPFS protocol defines how files are stored, retrieved, and transmitted in a distributed system, allowing for permanent and decentralized preservation and sharing of files. This is a content-addressable, peer-to-peer distributed protocol. IPFS has a very grand vision, hoping to serve as the storage layer of Web 3.0 and build a new internet architecture. Currently, more than 5 billion files have been uploaded to IPFS, and more than 100 blockchain projects have adopted IPFS for data and file storage, making it one of the important infrastructures of decentralized networks.

Filecoin is an incentive mechanism and public chain system based on the IPFS protocol. It has established an open, decentralized storage network to store data from all over the world. It serves as both the storage layer of IPFS and the incentive layer of the IPFS protocol, with IPFS being the application layer of the entire system. Both Filecoin and IPFS are developed by Protocol Labs, and the two protocols share multiple functional modules.

The Filecoin network ecosystem mainly consists of independent miners (participants who provide network storage capacity) and storage service customers who hire miners. Miners participate in network governance and compete for block packaging rights by providing storage space as effective storage power.

Miners are key contributors to the development of the project ecosystem, and the investment of token holders in miners and developers is also crucial for the growth of the ecosystem.

Filecoin uses a hybrid consensus mechanism: Expected Consensus (EC) + PoRep (Proof of Replicated Data) + PoSt (Proof of Space-Time).

The hybrid consensus mechanism of Filecoin has two advantages over traditional consensus mechanisms:

Firstly, it has high network transaction efficiency. There is no need for a large amount of computation. Storage service providers can obtain corresponding rewards using their storage space, effectively preventing the waste of costs incurred by storage service providers.

Secondly, effectively prevent blockchain forks. Filecoin uses a pledge mechanism and storage space weight to enable storage service providers to choose the most effective chain, while also punishing storage service providers who engage in forks through a corresponding punishment mechanism.

5.2 Filecoin Token Economy

The ecological token of Filecoin is FIL, with a fixed maximum supply of 2 billion FIL.

70% of the total token supply (1.4 billion tokens) is allocated for rewarding miners. The Filecoin Foundation owns 5% (100 million), Protocol Labs owns 15% (300 million), SAFT investors own 7.5% (150 million), and other early investors own 2.5% (50 million).

The evolution of decentralized storage

Token Vesting Period:

Miners receive 25% of the rewards immediately, with the remaining 75% being unlocked linearly over 180 days. The block rewards obtained by unreliable storage nodes will be reduced and burned. The mining reserve has not yet been released and will be determined by the community on how to distribute. According to the data from Filescan on August 23, 2022, the daily output for miners is approximately 290,000 tokens.

5.3 Analysis of Filecoin’s Prospects

Filecoin competes directly with major players in the traditional cloud storage industry, aiming to provide a cheaper alternative, disrupting the centralized cloud storage industry dominated by large companies such as Amazon, Google, and Alibaba.

To succeed in the cloud storage market, Filecoin needs to be competitive in terms of price. Even though Filecoin may be able to charge a certain premium in the long term due to the security and decentralized nature of its network, large centralized companies still have a significant competitive advantage in terms of economies of scale and can continuously lower prices. In the short term, it is still challenging for decentralized cloud storage service providers represented by Filecoin to overtake the competition.

Competitive Advantages:

The project has a large scale and abundant funds, with a complete ecosystem that is conducive to expanding the application scale.
In terms of the Filecoin mechanism, the entire network compensates miners for their hardware costs through block rewards to provide storage services. The storage services are traded through a bidding matching mechanism, which is beneficial in reducing the price of storage services.

Competitive Disadvantages:

The hardware requirements for Filecoin mining are relatively high, making it impossible for ordinary users to participate in mining with their personal computers. There is a risk that the entire network will tend towards centralization, with major data centers and server farms dominating.
In terms of the current download speed of the IPFS network, to meet the requirements of enterprise-level services, its performance and speed still need to be further improved.

Filecion analysis of the trackhead project

6.1 Arweave

Arweave was established in 2017, and its mainnet went live in June 2018. It is a protocol that focuses on a one-time payment for permanent file storage. Arweave provides a storage solution called Permaweb, a permanent web, which leverages the immutable nature of blockchain technology to directly write content into blocks for storage, thereby achieving true permanent data storage for the first time.

Currently, the internet loses 95% of its information every 20 years. Arweave addresses the issue of how to store data reliably for the long term by using Proof of Access (POA) to incentivize miners to store history indefinitely and share it upon request. Miners are rewarded not only with new block rewards but also with rewards for storing random old blocks in the chain. This means that the more blocks a miner stores, the more rewards they receive.

Arweave’s revenue in the second quarter of 2022 was $193,000, a 31% increase from the first quarter.

Project Advantages:

Arweave offers a one-time payment for permanent file storage, filling a market gap. Fundamentally, Arweave addresses the current issues of restricted freedom of speech, excessive censorship, and susceptibility to tampering on the internet.

At the same time, Arweave can also help organizations store complete and tamper-proof knowledge and information, such as storing a climate change database. Currently, Arweave is working with the National Oceanic and Atmospheric Administration (NOAA) in the United States to permanently store ongoing carbon dioxide measurements to test the Permaweb.

Arweave claims that it will provide data storage costs that decrease exponentially, similar to Moore’s Law. Users only need to pay a one-time upfront fee (about half a cent per megabyte), and the interest generated from overpayments will help to continue reducing subsequent storage services.

Arweave’s founder, Sam Williams, said, “In recent years, the cost of data storage on Arweave has decreased by an average of 30% per year. As long as this rate does not fall below 0.5% (based on today’s storage requirements), the decentralized network can still afford the costs, and today’s payments will cover the cost of storage space for the next 200 years.”

On the technical side, Arweave’s mining uses the RandomX algorithm and introduces a parameter for block completeness on this basis. Due to the characteristics of Arweave, its block size will increase exponentially with the use of the network. Considering the ever-expanding block data, there is a risk that future nodes may not be able to store complete block data. Therefore, the introduction of the block completeness parameter does not require all nodes to store complete block data, but instead competes for block packaging based on the number of blocks stored by the nodes.

Project Disadvantages:

Arweave’s unique features can be applied to data preservation mainly in HTML5 web pages, establishing decentralized H5-APPs. However, in actual use, we found that the application scenarios for this notarization are relatively narrow. Currently, the most stored content on Arweave is screenshots of some anti-government tweets. The increase in explicit anti-government applications is concerning.

At the same time, Arweave’s characteristic of being permanently unalterable makes it particularly challenging in program development. Programs uploaded to Arweave by developers must be free of any errors. If there is an error, even a single punctuation mark, the previously uploaded content will be invalidated and need to be re-uploaded, inevitably leading to a large amount of useless garbage accumulation. In addition, due to the openness of the blockchain, the content uploaded to Arweave is open to the entire society and is not suitable for uploading personal content.

Finally, Arweave focuses on a one-time payment for permanent file storage, which is a relatively simple model. However, there is a certain risk that homogeneous projects may emerge, utilizing the same storage concept and initiating a price war.

6.2 Storj

Storj was founded in July 2017 and is a distributed cloud storage protocol based on Ethereum, developed by the for-profit company Stroj Labs.

Storj aims to build a decentralized cloud storage platform that is free from censorship, surveillance, and downtime. By utilizing idle hard drives and bandwidth, any node on the P2P network can negotiate, transfer data, verify data integrity and availability, retrieve data, and store it.

In the second quarter of 2022, Storj’s revenue was $48,000, a 64% decrease from the first quarter. Its main focus is on enterprise-level storage services, with a more commercial model, directly competing with Amazon’s S3 service. Storj has also established a partnership with Microsoft Azure, with the hope of providing enterprises with storage services that are comparable, or even superior, to Amazon’s storage services in terms of various performance indicators.

As of now, Storj has over 13,000 nodes and is renting out its network to thousands of users. Recently, Storj has significantly improved the performance of video storage and management. If the project progresses smoothly, Storj will become one of the most commercially competitive decentralized storage platforms for enterprises.

Users can purchase storage services on the Storj platform using their platform tokens, $STORJ, a model similar to Airbnb and Uber. Users provide storage services using their idle storage space and receive $STORJ in return.

Most small devices, such as NAS, personal computers, and Raspberry Pi, can install a Storj node to rent out their idle space, which is suitable for building an extremely decentralized cloud storage network and is suitable for civilian mining. Storj claims to be the Uber of decentralized storage, capable of reorganizing idle social resources into usable commercial products.

6.3 Stratos

Stratos – The next generation decentralized data network.

The Stratos network provides decentralized storage solutions and launched its storage testnet in February this year, with the mainnet expected to go live later this year. Stratos is based on a powerful decentralized platform and a high-throughput native blockchain, and has received investments from institutions such as FundmentalLabs, Kenetic, Fenbushi Capital, and Assembly Partners.

The Stratos network is composed of four modules and three layers, providing the most foundational infrastructure for developers. The four modules are blockchain, decentralized storage, decentralized database, and decentralized computing.

The three tiers are the value layer, the resource layer, and the metadata routing layer, each associated with a different consensus method. The value layer uses Proof-of-Stake (PoS) consensus, the resource layer uses Proof-of-Traffic (PoT) consensus, and the metadata routing layer uses Proof-of-Authority (PoA) consensus.

Analysis of other events on the track

6.4 Ceramic

Ceramic is a decentralized, open-source platform for creating, hosting, and sharing data. It was formerly known as 3Box, an Ethereum-based user identity data protocol. Ceramic provides computation, state transition, and consensus synchronization for all types of data structures stored on a decentralized network, enabling the extraction of mutable dynamic data from static and immutable data on storage protocols.

Ceramic aims to address the more apparent shortcomings of IPFS, which is that once a file is stored, it cannot be updated in real-time and requires manual synchronization of dynamic data, resulting in low efficiency.

Ceramic’s data stream process allows developers to build secure, trustless, and censorship-resistant DApps based on dynamic information without the need for a trusted database server.

The core components of Ceramic include three elements:

A scalable, decentralized data infrastructure;
Open APIs for storing, modifying, and retrieving data;
A marketplace for reusable data models created by the community.

6.5 Crust Network

Crust Network is a parachain of Polkadot, designed to provide decentralized cloud storage services. As an important storage-class public chain in the Polkadot ecosystem, Crust Network is expected to become an important part of the Web3 infrastructure construction. The unique design of Crust Network mainly has two aspects: MPoW (Meaningful Proof of Work) and GPoS (Guaranteed Proof of Stake).

MPoW effectively solves the problem of reporting node workload, characterized by transparency, fairness, efficiency, and development. In simple terms, the storage mechanism is simple and transparent, the workload of miners is directly proportional to their rewards, storage space is efficiently utilized, and the development of TEE has unlimited potential. Under the GPoS consensus mechanism, Crust Network’s ecosystem has produced four roles, namely, validators, candidates, nominators, and users.

Validators are nodes in the network that package and generate blocks, maintaining the entire blockchain network, requiring the storage of assets as collateral, and needing to be online in real-time, similar to the miner nodes in the Filecoin network.

Candidates are nodes that compete to become validators but ultimately do not have the qualifications to be validators, also requiring the storage of assets as collateral, and needing to be online in real-time. However, the role of candidates is not fixed, and they have the opportunity to become validators.

Guarantors are accounts in the network that provide guarantees for any one or multiple nodes, and can earn guarantee income by providing guarantees for nodes.

Users are the main entities with storage needs, purchasing storage services by paying CRU and other tokens of the Crust Network, and using the network’s storage space.