Essentials
Getting Started
Artist Guides
Decentralised Storage: IPFS and ONCHFS
An overview of the different storage solutions that fxhash integrates to make available the projects uploaded to fxhash and their associated outputs.
One of the core principles of Web3 is decentralization: a paradigm that introduces many benefits over the previous iteration of the web, such as a censorship resistant environment without central authorities that users have to rely on, trustless environments with improved security, and the provision of users with greater control and ownership over their data.
In other words, Web3 envisions a more decentralized and user-centric internet, leveraging blockchain and decentralized technologies to empower individuals with control over their digital assets, including data.
Decentralization in the context of Web3 means distributing control and authority across a network of participants rather than relying on a single central authority, like a traditional server or company. In a decentralized system, data and processes are spread out among many nodes or computers, making the network more resilient, resistant to censorship, and less dependent on any single point of failure. This shift toward decentralization in Web3 aims to empower individuals, giving them greater control over their data, identities, and interactions online, and reducing the dominance of centralized entities in the digital landscape. In this setting, it doesn’t make sense anymore to have data stored on a central server owned by an entity or company. Centralized storage solutions need to be replaced by a different decentralized method.
Because Data cannot exist in vacuum and actually still needs to be physically stored somewhere, several solutions have been devised for this purpose. fxhash leverages IPFS and ONCHFS.
IPFS
IPFS is an abbreviation for Inter Planetary File System. In essence, as the name suggests, IPFS is a file system designed for the storage of various types of files. It functions vastly differently from the regular file systems that one might be used to. Here, the phrase "Inter Planetary" is a play on words that symbolises the decentralised character of this file system. Decentralization being a core value of Web3 technology makes IPFS a good fit and has therefore received wide adoption.
A good introduction to IPFS in video format can be found here:
A Decentralized Peer-to-Peer Network
Traditionally, in Web2.0, files are stored on centralized servers, centralized in this setting means that these servers are owned by specific entities, companies and individuals. Although users can view and interact with that data via the web, they are always reliant on this central authority to serve that data.
Instead of relying on such a centralized approach to file storage, IPFS uses a decentralized approach that distributes files over a network of nodes - where these nodes are simply the machines of the users that choose to participate in the network.
Moreover, in this setting, each user's device on the network is both a client and a server. This is in contrast to the client-server model where users connect to a central server to request data. With IPFS, users can request and share data directly with each other. Peers on the network collaborate to share and distribute files, creating a more resilient and scalable system.
Content Based Addressing
Traditionally, when requesting a file from a server we are essentially asking the server to retrieve his file from a certain location by means of an address (urls basically), this is known as location based addressing.
Instead of location based addressing, IPFS makes use of content based addressing. This is because the same file could be stored on several nodes throughout the network, hence we would simply need to retrieve that file from one of these nodes. In this case it doesn't matter where the file comes from, simply that the correct file is requested. This can be done by assigning a unique identifier (a hash essentially) to each file on the network.
Files and all of their versions are uniquely identified using content-addressed hyperlinks, meaning that the content of a file is used to generate its unique identifier (hash). This hash is then used to locate the file on the IPFS network. This hash isn't generated at random, but rather is derived from the data itself, meaning that the data predicates what this hash will look like, making it such that there is a 1-to-1 mapping between the hashes and the files.
Having the hash derived from the data itself has several inbuilt advantages:
- Security: when requesting a file by its hash, the data needs to match the hash otherwise it is not valid. This makes it such that Files can't be tampered with by actors on the network.
- Deduplication: If two files are identical, they will produce the same hash, meaning that there is no reason to store the file a second time.
- Immutability: Once a file is added to IPFS and given a hash, the content cannot be changed. If you modify the file, you create a new hash. This ensures data integrity and allows anyone to verify that the content they retrieve matches the expected hash.
- Versioning: The technical details of how this is implemented are beyond the scope of this resource, but IPFS also makes it easy to update files by providing a versioning system. Every change to a file results in a new identifier, allowing for easy versioning and tracking of changes, where the new file adds a pointer to the previous one.
Downsides of IPFS
While IPFS has many advantages and innovative features, it's important to be aware of some potential downsides and challenges associated with the technology. Some of them are:
- Content Availability: most notably the biggest flaw of IPFS, for files to be available on the network, there needs to be at least a single node hosting that file and making it available. If this node for some reason goes offline, as a consequence, the files that are only hosted by this node also become unavailable.
- Speed: The speed at which you can retrieve content from IPFS depends on the popularity of the content and the number of nodes (peers) hosting it. Less popular content might take longer to retrieve, and some content may not be available if there are not enough nodes hosting it.
One countermeasure to content becoming unavailable is "pinning" (from the verb to pin), where the IPFS network is explicitely instructed to retain a specific piece of content regardless of it being accessed by nodes or not. There are certain dedicated services for this purpose than can automate this pinning process, such as ClubNFT, but can equivalently be done from your own local machine, which would require a little setup however.
How does IPFS integrate into fxhash?
IPFS is often used for various applications, including decentralized websites, file sharing, and data distribution. It aims to create a more resilient and censorship-resistant internet by changing the way data is stored and retrieved, making it less dependent on centralized servers, thus making it a popular solution in that regard.
ONCHFS
ONCHFS stands for On-Chain File System. ONCHFS is a permissionless unix-like content-addressable file system fully stored on-chain designed to be delivered through the http protocol, with cross-blockchain compatibility and extensibility in mind.
That's a lot to take in at once, let's address each point individually:
- Permissionless: Anyone can write data, create files & directories, & reference files stored on the file system.
- Unix-like: this simply means that onchfs is inspired by the unix-file system, where data is organized in a hierarchical tree-like structure of directories and files.
- Content Addressable: content is addressed by unique identifiers derived from the data itself.
- On-Chain: files are entirely stored on the blockchain, making them immutable and persistent as long as the respective blockchain is alive.
- Designed for the http protocol: the ONCHFS protocol makes it possible to address and retrieve the stored assets with the http protocol that powers the modern web.
- Cross-Blockchain: ONCHFS works on any chain, and files stored on one chain can reference files stored on other chains.
- Extensibility: ONCHFS is built in such a manner that it can easily be extended and other technologies to be built on top of it.
ONCHFS is inspired by ethfs, ipfs, bitcoin ordinals recursive inscriptions, Unix filesystem, and aims at providing a general framework for working with files stored on-chain, following modern standards & practices.
Link to revdancatt’s vid
Why store files on-chain?
Before we dive into the specifics: why would we want to store files fully on-chain?
Ever since NFTs have become more popular, the on-chain vs. off-chain discussion has become more and more heated. One cherished aspect of blockchain technology is longevity, security, and immutability, especially when it comes to transactions which play a big role in making generative art as collectible digital tokens possible. In the previous page, we’ve covered the IPFS storage solution, which has characteristics that closely align with Web3 technology, in turn making it a popular solution for storing content and data in the setting of NFTs. Towards the end, we learn that it also introduces an element of impermanence, which requires additional effort from artists, collectors, and platforms to ensure the availability of the content.
This could in theory all be solved by storing files directly on-chain. But there are two problems that come in tandem with this approach, firstly, the high cost of uploading and eternalizing large data loads on digital ledgers, and secondly, the absence of a good system or protocol for this purpose.
In the setting of generative art, the first problem solves itself - the actual image data (that represents the art) doesn’t need to be stored directly on the blockchain, merely the code that can generate the art. Code being simply a bunch of alphanumeric symbols, is highly compressible and can be shortened enough to not introduce exorbitant minting costs. The second problem is addressed by ONCHFS which is introduced in greater detail in what follows.
Overall, storing files on-chain is preferred by many because it is considered a purer and more persistent method for storing files that ensures the existence thereof for as long as the blockchain exists. And for the popular blockchains there is no worry for them to simply disappear overnight.
What problems does ONCHFS solve?
Previous attempts at on-chain storage solutions have had some inherent problems that ONCHFS tries to address. For instance, in the context of generative art projects that come in the form of self-contained web projects which the browsers can run, existing solutions don't actually fully store the entirety of the projects on the blockchain, but rather only a part of them - generally the code that creates the artwork. Generally, this code is minified and compressed into a string for it to be efficiently stored on the blockchain.
This is problematic however, because it makes it such that platforms often need to implement an additional off-chain step to reconstruct the project as a whole for it to actually run in the browser. The javascript code that generates the artwork can't run in a vacuum and needs additional files to be complete, such as an html document that serves as an entry point, a stylesheet to specify its appearance in the browser, and additional other javascript libraries that it makes use of and builds upon.
Recreating this project structure is referred to as html-reconstruction where the platform creates the index entry point that serves and displays the project in the browser. Additionally the platform needs to provide and host a specific list of dependencies that the project requires to function properly, this doesn't align with the decentralised ethos of Web3.0.
ONCHFS solves both of these issues, projects are entirely stored on-chain such that no html reconstruction step is required. Additionally libraries, dependencies, and other assets can also be freely created by users, making it possible for anyone to reference them. In this manner, it liberates code artists in their creative endeavours, ensures longevitiy of projects stored with ONCHFS, and opens the door for interesting and creative projects that make use of this on-chain file storage paradigm.
How does ONCHFS work?
ONCHFS is designed for the purpose of preserving the integrity and longevity of generative art projects, making it excel in that regard. The ONCHFS protocol ensures that uploaded projects are delivered in a 1:1 format, without requiring any additional steps to reconstruct the original project.
It also handles project dependencies in an elegant manner. Storing files in a content addressable manner, where each file has a unique identifier derived from the data itself (we cover this in a bit more detailed manner in the page on IPFS), makes it possible to detect if a library already exists on-chain. In this case, the library doesn't need to be re-uploaded a second time, and it will automatically link the already existing stored version. ONCHFS handles this by creating the content ids at upload time.
This introduces a paradigm of trustless and fully open resources sharing, as no one has to rely on a centralized platform to provide a library, or a set of personal tools they keep reusing over time; if they upload it once, it will become available forever for free for everyone using onchfs.
What's more, is that the resources stored with ONCHFS are http compliant.
HTTP, or Hypertext Transfer Protocol, is the foundation of data communication on the World Wide Web. HTTP facilitates the transmission of various types of data, including text, images, and multimedia, using a request-response model. When a user enters a web address or clicks on a link, the browser sends an HTTP request to the server, specifying the desired resource. The server then processes the request and returns an HTTP response, containing the requested data or an error message.
This means that when files are uploaded to ONCHFS, they are also formatted in an http compliant format that can be transferred by means of the protocol. Naturally, the artist/user uploading to ONCHFS doesn't need to worry about any of this, it is automatically handled ia the fxhash minting interface. It is possible however to also manually interact with ONCHFS.
A complete documentation of ONCHFS can be found here:
Chain Agnostic
One innovative aspect of ONCHFS is that it can be used to store files on any blockchain, without having their existence isolated and restricted to that particular chain. This means that a file on the one chain can reference files stored on other chains.
This can become useful if there is a need to use a cheaper blockchain for storing bigger chunks of data to be accessed more expensive blockchain. While not a typical scenario, it's worth noting the built-in onchfs support for such cases.
Limitations
The main limitation of ONCHFS is that it is a relatively new method for storing files on-chain, meaning that it isn't as widely adopted as other decentralized storage solutions such as IPFS. With fxhash growing as a platform and community and with more files being stored with ONCHFS it is likely to become adopted by other platforms that would like to display these generative art projects.
← Previous
Next →