
In the past two years, the NFT craze has swept the world. Minting NFTs on various blockchains has become commonplace. However, the storage of these NFT assets is often overlooked. IPFS, also known as the Interplanetary File System, has become a common standard for storing NFT assets.
An often overlooked detail is that IPFS itself is not a storage network. It's more of a data routing and transport protocol. The IPFS network is a collection of nodes that exchange information. File entries can be "pinned" to a public IPFS DHT (Distributed Hash Table), letting other nodes on the network know which hosts are storing which files.
In the IPFS ecosystem, there are many public gateways. Some of these gateways allow you to upload files, but there is usually no guarantee that your files will remain online. Unless the file is explicitly pinned by an IPFS node, the file will be deleted the next time the garbage collection process runs. This exposes some issues with NFT storage.
To help with this, a few different pinning providers have emerged. For a fee, these providers allow you to upload files, and they offer to pin them for you.
However, this brings up a new question, where are these files actually stored? The developers spent a lot of time researching this problem, and through the analysis of nearly 1 billion files, they found the following problems:
1. Most IPFS pinning providers use Amazon S3 and other centralized object storage services in the background. The "data store" of an IPFS server can be pointed to S3 using a simple plugin.
2. Since I am using AWS S3, the provider is charging upwards of $150 per TB!
3. If AWS S3 fails, your IPFS server will also fail. Unless the data has been cached somewhere, your IPFS CID link will no longer be accessible. This can lead to issues like "rug pull" with NFTs.
4. The system with this configuration also introduces a very peculiar way to access AWS S3. If we're so dependent on AWS, why not just use a simple HTTP URL?
The developers have come up with a solution to the problems listed above and will benefit the entire crypto ecosystem. Simply put, build Web3 with Web3.
All files pinned to IPFS using Filebase are actually stored on Sia, one of the leading decentralized storage networks. This creates an environment where the data storage layer of IPFS nodes is highly available and, most importantly, geographically redundant. By using a decentralized network for storage, we no longer depend on cloud providers for block storage volumes (AWS EBS) or centralized storage buckets (AWS S3).
How to pin data on IPFS? Easy, you have two options:
1. Use the simple drag and drop interface in the Filebase dashboard
2. Use a simple S3-compatible API
When you upload a file, an IPFS CID (Content Identifier) will be returned. You can then use this CID to access your data from your own IPFS node or any other IPFS gateway on the public internet. CIDs are clearly displayed in the dashboard, and they can also be retrieved programmatically.
After opening an IPFS bucket from the dashboard, there will be a CID column. You can click on any CID and it will be automatically copied to the clipboard.
The object overview page will also show you the CID along with other details:
If you are using an S3-compatible API, the CID will be returned in the response to the PutObject call. For example, if we run the following AWS CLI command:
The response is shown below. For convenience, the corresponding response headers are highlighted
We can also call the HeadObject API at any time to get the CID:
Now that the IPFS functionality is live, the developers plan to continue building this integration further!