At ThreeFold, we believe that everything we do should benefit people and our planet. The emergence of a decentralized Internet grid not only reshapes the digital landscape but helps to pioneer a more sustainable future. At the core of this evolution lies a series of innovative strategies poised to revolutionize the way we perceive and utilize digital infrastructure.

🔗Zero OS

Traditionally, operating systems have been laden with an array of features, consuming substantial hardware resources. However, with Zero OS, our paradigm shifts towards a lightweight, autonomous design. By getting rid of unnecessary complexities, this design not only optimizes hardware space but also fosters an environment conducive to enhanced IT workloads.

🔗Quantum-Safe Storage

One of the most staggering inefficiencies within the industry has been the substantial overhead of storage systems. Our quantum-safe storage system marks a monumental departure from the norm, slashing the overhead from a staggering 400% to a mere fraction, 20%. This unprecedented leap in efficiency heralds a new era of streamlined storage practices, ensuring optimal utilization of resources.

🔗Efficient Node Management

Energy conservation takes center stage with our node management tool, Farmerbot. Embracing the principle of ‘on-demand’ functionality, nodes remain dormant when idle, conserving energy resources. Through the implementation of Farmerbot, these nodes seamlessly power on and off as required, minimizing unnecessary consumption without compromising accessibility.

🔗Automated Autonomy

We’re saying goodbye to human intermediaries and centralized control centers. Our system operates autonomously, governed by smart contracts that orchestrate every facet of its functionality. This automated ecosystem not only eradicates the need for constant human intervention but also ensures streamlined operations, optimizing efficiency at every turn.

🔗Peer-to-Peer Advancement

The conventional route from sender to recipient often travels extensive distances, bouncing through data centers and intermediaries. Our peer-to-peer network revolutionizes this process by forging direct connections, seeking the shortest path between sender and recipient. By circumventing unnecessary detours, this approach minimizes energy expenditure and data latency, elevating both speed and efficiency.

🔗Final Thoughts

The dawn of ThreeFold’s decentralized Internet grid beckons a transformative era in digital infrastructure. Through the combination of a lightweight operating system, quantum-safe storage, energy-conscious node management, automated autonomy, and a peer-to-peer network, the paradigm of efficiency in the digital realm has been redefined. The aggregated result is up-to 90% energy savings depending on workloads. This evolution not only champions sustainability but also propels us towards a future where connectivity thrives in harmony with resource conservation.

🔗Yggdrasil’s Limitations

To start, it should be noted that Yggdrasil is an experimental routing protocol (based on DHT’s), which is currently still in an alpha stage. We very much admire the fact that they have been able to create a standalone network with worldwide participants from scratch. Over time, the network has grown, in part due to the grid, but also because a lot of individual participants joined and started running a node.

After investigating, we reached the conclusion that the current architecture has some scalability issues. While we have no doubt that these will eventually be solved, the importance of the planetary network in the grid is unfortunately too big to be able to wait for this. Additionally, Yggdrasil uses a technique called crypto key routing, and uses a tree structure for its routing info. This means data follows the “shortest path” in the tree, rather than the physical shortest path, which might deviate from this. As such, we have decided to implement our own Ipv6 overlay network, Mycelium.

🔗Implementing Mycelium

As previously pointed out, the main goal of our network is to be efficient, and end-to-end encrypted. Every node in the network is identified by a key pair. The address of a node is derived from this key pair, and data traffic between 2 nodes is encrypted using a secret derived from the key pair of both these nodes. As a result, every 2 nodes in the system use a different secret for encryption, and we can verify that the address used by a node is indeed owned by it (i.e. it has the corresponding private key). We have based our initial implementation (loosely) on the babel routing protocol.

Therefore, sending data to a remote node will follow the shortest path in terms of latency (i.e. the fastest route). The network adjusts itself in case nodes disappear or join, and multiple paths to a given destination are possible.

We are also considering implementing more advanced use cases in the future, though this remains to be seen at this stage. We are gearing up for some larger scale testing on the grid in the near future (after all, we do have multiple thousand nodes at our disposal to verify scalability), and will adjust as needed once these are done.

🔗Rising Energy Prices

Energy prices have been on steady rise globally because of increasing costs of energy production, high demand, and the depletion of traditional energy sources such as oil and natural gas as primary drivers. However, the Russian-Western standoff due to the invasion of Ukraine has produced an unprecedented increase in energy prices, especially in Europe.

Some statistics:

• UK - Household energy bills increased by 54% in April 2022, and a further increase of 20% is expected in April 2023.
• USA - Consumers paid 14.3% more for electricity last year on average.
• France - As of February 2023, electricity prices may rise by 15%.

These rising energy costs have a significant impact on the economy, affecting inflation and driving up the cost of goods and services.

🔗Reducing Our Bills & Our Impact on the Environment

Given the increasing cost of energy, it is more important than ever to adopt responsible energy use practices. Simple measures like turning off lights and electronics when not in use, using energy-efficient appliances, and properly insulating homes can make a big difference in reducing energy consumption and lowering bills.

By reducing energy waste, individuals and businesses can not only save money, but also help protect the environment and preserve natural resources for future generations. The world’s energy resources are finite, and the increasing demand for energy is putting pressure on the environment. By adopting responsible energy use practices, we can reduce our carbon footprint, decrease our dependence on non-renewable energy sources, and preserve the planet for future generations.

Saving energy is important for the planet as it can have a positive impact on the environment and the economy. Some of the key benefits of energy conservation are:

• Reduces greenhouse gas emissions:
  • By reducing the amount of energy we consume, we also reduce the amount of greenhouse gas emissions produced by power plants, which are a major contributor to climate change.
• Saves natural resources:
  • Conserving energy helps to conserve our limited natural resources, such as coal, oil, and natural gas, which are used to generate electricity.
• Lowers energy costs:
  • By using energy more efficiently, we can lower our energy bills, saving money and reducing our overall energy costs.
• Improves energy security:
  • By reducing our dependence on foreign sources of energy, we can improve our energy security and reduce the risks associated with energy shortages and price spikes.
• Creates jobs:
  • The development and implementation of energy-efficient technologies and practices can create jobs in the energy efficiency and renewable energy sectors.

While it has been known for a long time that consuming less energy meant a smaller ecological footprint, most people weren’t very motivated to do so. However, once their wallets started feeling the price increase, people became increasingly aware & careful of their energy consumption. So, in a way, while it is not a great thing for our economies, the rising energy prices are doing a good thing for the planet.

🔗Energy Usage of Nodes

The energy usage of a node in a network depends on various factors such as the type of node, its hardware specifications, and the workload it is handling. Generally, nodes that perform more intensive computations and have more advanced hardware specifications will consume more energy compared to basic nodes.

For example, nodes used in cryptocurrency mining operations are known to consume large amounts of energy due to the intensive computations they perform. On the other hand, simple nodes used for basic network functions such as routing or data storage may consume less energy.

It is important to note that the energy consumption of a node can also be influenced by environmental factors such as temperature, network congestion, and power supply efficiency. By optimizing these factors, it is possible to reduce the energy consumption of a node and minimize its environmental impact.

🔗ThreeFold 3Node Servers

ThreeFold is a decentralized internet infrastructure that utilizes a network of nodes (referred to 3Nodes on our grid) to provide computing resources and data storage. Farmers can farm capacity for the ThreeFold Grid by building their own server and hosting Zero-OS on it, or by buying and running a preconfigured device.

In general, a 3Node cluster is expected to consume less energy compared to traditional server clusters, due to its focus on energy efficiency and sustainability. However, the exact energy usage will depend on the hardware specifications of the nodes, the workload they are handling, and the energy-saving measures implemented. It’s important to note that even a machine with nothing running on top of it consumes energy and brings a cost to farmers.

🔗ThreeFold WOL Feature Comes as Part of Our Farmerbot

Wake-on-LAN (WOL) is a feature in computer hardware that allows a network administrator to power on a computer remotely. The feature works by sending a special “magic packet” to the target computer, which contains the Media Access Control (MAC) address of the network adapter. The network adapter is programmed to recognize the magic packet and use it to initiate the power-on process.

Wake-on-LAN is useful for remotely accessing a computer that is turned off, as it allows the computer to be powered on without physically pressing the power button. This feature is commonly used for remote administration and maintenance tasks, as well as for remotely accessing a computer for work or entertainment purposes.

Wake-on-LAN requires specific hardware and software support, including a network adapter that supports the feature, a motherboard that enables the feature in the BIOS, and an operating system that supports WOL. Additionally, the computer must be connected to a network that supports WOL, and the feature must be enabled in the network adapter’s settings.

The community has been asking for an energy-saving solution for quite a while and we had been looking for one. It took some time, as this was not the only topic that needed our team’s attention, and we had obstacles along the route to designing this solution, including limited funding and needful adjustments.

We had been working on a WOL feature for power-saving set up directly on the chain, but it turned out to be a dead end. Instead of an independent WOL feature, our dev team started from scratch with a new idea! We are now very close to delivering a solution: A Farmerbot that uses WOL to switch nodes on and off when needed!

Farmers will be able to define their configuration in an easy markdown format. The Farmerbot will check which nodes can be turned off or not. Farmers will be able to personalize this feature according to their farm’s own needs and preferences.

🔗Conclusion

Saving energy is crucial for the planet and has numerous positive impacts on the environment, economy, and society. By taking steps to conserve energy, we can help to reduce our environmental impact, lower energy costs, improve energy security, and create jobs in the clean energy sector.

Having a power-saving feature for ThreeFold nodes is one of our priorities. Together with the community, we agree that this is a pressing issue we need to solve. Our main incentive is to respect our engagement to be a sustainable network that puts the planet first. Supporting our farmers to get through difficult times financially is an extra one.

Similar to the enterprise blockchain hype that happened a few years ago, Web3 seems to be repeating the same story, with lots of chatter but lagging adoption. Millions have rushed to trade crypto and NFTs attached to Web3 – with a lot of this activity based on speculation and hype – but that doesn’t mean they will replace legacy platforms anytime soon. New approaches such as DeFi (decentralized finance) don’t have the same problem of the legacy firms, but they face the challenge of scaling and mass adoption. However, many so-called ‘Web3’ solutions are not as decentralized as they seem, and still have to show they are scalable, secure and accessible enough for the mass market. This all may change, but there are clear limitations that will need to be overcome.

Whether Web3 is a marketing slogan or a true technological trend, there is a lot of money and technology behind it. Financial institutions are embracing blockchain technology and we are seeing a rising adoption of digital assets across industries. While the idea of utilizing digital currencies to exchange different forms of value in an increasingly digital world certainly makes sense, it’s a bigger leap to say that blockchain alone can support a broad set of use cases, including replacing the current Internet infrastructure.

The Internet’s client-server infrastructure replicates data and workloads across many computers, and uses tons of computing resources and energy while limiting the scalability of our applications. However, the clear inefficiencies and gaps in security in today’s centralized infrastructures are being challenged by simpler means of building, storing and exchanging data.

Blockchain is a clear evolution to the centralized client-server model, but there are severe challenges sourced from the blockchain trilemma between decentralization, scalability and security. Peer-to-peer systems transcend those challenges as they don’t require any valid2ation or verification from a third party, and enable end-to-end connections that can scale limitlessly without the potential of being altered or corrupted by intermediaries. In peer-to-peer systems, all participants are equally privileged, presenting a fairer model. Peer-to-peer represents an incredibly simple and efficient paradigm for organizing our digital world, and has already proven to be the most efficient alternative to centralized information systems.

Simply put, there is no reason to replicate full sets of data and workloads across many computers, and use tons of computing resources and energy while limiting the scalability of our applications. While there is an increasing reliance on blockchain principles to organize our Internet systems, blockchain also makes things redundant, slow, complex and energy intensive. To be clear, blockchain isn’t the problem, as it is a great technology to store and exchange value in a decentralized way, but it is only one piece of the puzzle.

By inter-connecting the computing processing power and storage capacity of billions of computers worldwide in a peer-to-peer way, data transfers could happen end-to-end between people and machines without the need to be verified. Peer-to-peer is also the only way to achieve a truly self-sovereign Internet – creating a ‘trustful’ environment that allows users and developers to have real data ownership.

Peer-to-peer systems aren’t meant to replace blockchains – they represent a way to decentralize models that blockchain cannot, such as Internet infrastructures. Peer-to-peer systems and blockchains, side by side. The best of both worlds. Peer-to-peer as the most efficient and decentralized alternative for us to exchange data and Internet resources. And blockchain for establishing trust through an irrefutable record of transactions.

With the rise of emerging technologies, a new technological foundation is needed to protect our sovereignty and freedom. The rise of blockchains, while well intended, ultimately has made the technological space more complex than ever before, and still benefits a plutocratic few (relative to its promise). More than just technology, peer-to-peer opens a world of new possibilities for humanity to co-initiate new paradigms and experiences.

By applying blockchain and peer-to-peer technology to the cloud, ThreeFold solves the security and autonomy issues of the Internet and realizes the initial promise of the Internet as an open-source, peer-to-peer, planetary mesh network of nodes owned by the people.

After introducing the ThreeFold Grid in part one and diving deep into storage and compute, we’re back with part 4 to look at the networking aspects of the ThreeFold Grid.

Computer networking is a rather complex phenomenon with a thankfully intuitive outcome: you send messages from one place to another. We all understand that our digital devices become a lot less interesting when they are not connected to the Internet. However, networking is not just Internet, and when it comes to building a new Internet, it’s important to understand what that means.

The existing Internet can be defined as a global set of networks and protocols that enable us to do things like load web pages and use apps that rely on external data. It depends on lots of underlying infrastructure, including fiber optic cables running along the ocean floor, our home routers and their connection to an Internet service provider (ISP), lots of cell service towers, and an increasing number of satellites. All of that makes the Internet possible, and it’s still required for a new Internet to function.

ThreeFold uses and builds new protocols to create a network that is interoperable with the existing Internet. These include private overlay networks, the Planetary Network based on Yggdrasil, and various interfaces to the public Internet. Let’s see how they work.

🔗Private overlay networks

Every deployment on the ThreeFold Grid belongs to a network, which is one of the Grid primitives. These networks allow data to be transmitted between workloads in a private and secure manner. They are called overlay networks because they pass traffic over some underlying network. That could be within a single node or between nodes on infrastructure provided by a farmer. The Grid uses software called WireGuard for this purpose, a simple, fast, and secure option among its alternatives.

By means of encryption, the traffic on a private overlay network is unreadable to any other device on that network. Likewise, the workloads within the private network have no visibility of the underlying network. Individuals who deploy workloads can access their networks over a secure tunnel, assuming that one of the nodes in the network is reachable from the public Internet. This is suitable for workload administration and some private applications. For public services and peer-to-peer communication, the ThreeFold Grid provides other solutions.

🔗Planetary Network

The Planetary Network is an implementation of the Yggdrasil Network, which is a next generation approach to network routing. It is a peer-to-peer solution where all traffic is encrypted and takes the shortest path to its destination. These features make it highly compatible with the ethos and goals of the ThreeFold Grid: privacy, security, and efficiency. The design also allows nodes, workloads, and users to communicate freely without relying on centralized intermediaries.

Many ThreeFold Farmers connect their nodes to a home router which blocks all inbound traffic by default. This is a sensible feature that doesn’t interfere with most Internet usage (enthusiasts who want to host their own website or a gaming server might bypass this, for example). However, nodes on the Grid need to be reachable, especially in order to receive workload definitions from deployers.

The Planetary Network solves this by accepting inbound traffic through an outbound connection to a publicly accessible peer. You can think of this like making a call to an operator and then waiting for another call to be routed to you on that line. Even if your phone is set to block all incoming calls, someone can still reach you if they can reach the operator.

This also provides a way for workloads to communicate with one another and be reachable by users. Anyone can enable the Planetary Network on their own device to access sites and services in a very secure and private way. While the ThreeFold Grid is designed to be fully interoperable with the existing Internet, the Planetary Network provides an alternative which can sit side by side in a seamless way.

🔗Reliable Message Bus

Using the Planetary Network, nodes and Grid users can send messages over the Reliable Message Bus. Notably, it is used for the delivery of workload definitions as mentioned earlier. Zero-OS supports some other RMB messages as well, for doing things like checking available capacity on a node in real time. RMB is a general purpose protocol that can be used by developers in their applications. It queries the ThreeFold Blockchain as a kind of address book for the Planetary Network, enabling nodes and users to find each other in a decentralized way.

🔗Public IPs and Web Gateways

Finally, let’s see how the Grid can offer services over the public Internet using public IP addresses and web gateways. Everytime we type a domain name into our browser, like threefold.io, it is resolved to an IP address. The browser then forms a connection with that address to retrieve the site we’re looking for. Reviving a theme from earlier, that IP must be publicly reachable.

🔗Public IPs

We’re in the midst of a shift from the older IPv4 standard to the newer IPv6 standard. While IPv6 has plenty of addresses for every device on the planet and many more, it hasn’t been universally adopted yet, and IPv4 addresses are in short supply. You likely have one assigned to your router at home, and these days, even getting a second one from your ISP can be a difficult and expensive proposition.

The ThreeFold Grid allows farmers who own blocks of IPv4 addresses to rent them out to deployers. These addresses get attached to individual workloads which are then reachable over the public Internet. A deployer could then assign their own domain name to that IP and host a website there. This is a very flexible way to make Grid deployments publicly available, but it comes at a relatively high cost.

🔗Web Gateways

Web gateways are another way to make workloads on the ThreeFold Grid available over the public Internet. In this case, the farmer may provide a domain name that points to one of their nodes which is publicly reachable. Deployers are then able to reserve a subdomain, like explorer.threefold.io is a subdomain of threefold.io, that routes traffic to their workload. It is also possible for a deployer to point their own domain to the gateway node which then routes traffic to the appropriate workloads.

We glossed over a bit in the earlier description of domain name to IP address resolution. In fact, multiple domains can point to a single IP address and this is common practice among existing web hosts. Adding similar functionality to the Grid helps to ease the demand for IPv4 addresses and is a lower cost solution for deployers than renting a dedicated IP. The web gateway also serves to enhance security and reliability of Grid-hosted services.

The gateway provides a connection from the public Internet to the secure private overlay networks that all workloads use. However, this is not a standard network connection but instead a network socket, which provides security through separation. This is something like passing along the contents of mail pieces without their envelopes. Web gateways also enable redundancy, as multiple gateways can point to the same workload, and multiple workloads can serve the same set of gateways.

When we say the ThreeFold Grid scales limitlessly, this is one of the ways that’s true. Many nodes, many gateways, many applications, and many users can all fit into this model. Deployers can start with a single virtual machine and a single public IP like in the example I’ll use to wrap up this piece below, then ramp up to a globally distributed and highly redundant system as their requirements grow.

🔗Send, receive, complete

Now we’ve seen how the ThreeFold Grid handles networking. From private and secure communication between workloads, to next generation peer-to-peer communication and full compatibility with the existing Internet, these offerings are comprehensive. Not only that, but they can be pretty darn fun to play with too.

In my own recent experience, I deployed a virtual machine on the Grid with a public IP and hosted a simple website with a map of nodes on the Grid that I’d been working on. I was able to quickly share it with some colleagues and members of the community. It went from a project I was tinkering with on my own computer to a live site on the Internet in a matter of minutes, thanks especially to the networking features of the ThreeFold Grid.

For the final piece of this series, we’ll see how everything comes together to turn computer code into a meaningful digital experience. Stay tuned!

🔗What is compute?

While storage and network are fairly intuitive concepts – where data lives and how it gets around – compute is about the more mysterious realm of data processing and ultimately computer programming. Don’t worry though, you don’t need any technical abilities to gain a deeper understanding of what’s happening behind the scenes when we use apps or interact with web sites.

As mentioned in the first part of this series, compute is basically what happens with data once it reaches its destination. Compute gives data relevance, by processing it according to the instructions provided by developers in their source code. When talking about what the ThreeFold Grid offers in terms of compute, it’s mostly about how the code that developers produce gets executed in order to provide the services we enjoy.

With ThreeFold, this actually happens for the most part using the same tools and systems provided by traditional cloud services, meaning that it’s highly compatible with existing cloud applications. What really makes the Grid different is that it’s been designed to provide a level of efficiency and security that simply isn’t possible without starting “from the ground up”. Before we get into specifics of compute offerings built into ThreeFold’s Zero-OS, let’s briefly explore what computing is.

🔗Playing with zeroes and ones

Everything a computer’s central processing unit does is based on a limited set of instructions that includes basic mathematical functions, logical operations, and ways to control the flow of a program. At their core, computers are really just automated adding machines that execute specified operations on whatever input they receive. It may be hard to believe that simple math creates all of the vibrant digital experience we enjoy, but at a certain low level, that’s the truth.

Take a messaging app as an example. Within a computer, all of the text, emojis, and images we see are represented by numbers. When we search for some text within a chat, matches are determined by checking if the numbers corresponding to the words we typed are equal to some numbers corresponding to words that appear in the chat history. This information is then converted into the list of results we see on screen.

In this context, compute is mostly responsible for organizing all of our messages based on the individual or group we were talking with, the time that the message happened, and whether or not we’ve opened it yet. Typically, a messaging app stores data both on our local devices and on a back end in the cloud. Compute is also how the service decides to copy messages you sent on one device to other devices where you have the same app installed.

Here is an important distinction about how the ThreeFold Grid operates. In a traditional cloud environment, it is impossible to know for sure what is happening on the backend, even if you are the one running the service. The administrative access retained by the cloud provider can be used for surveillance and tampering that can be very difficult or impossible to detect.

Zero OS offers a unique assurance that no one can abuse the Grid in this way, because it operates autonomously with no facilities provided for human intervention. In order to utilize compute capacity on the Grid, individuals and organizations deploy their choice of several “primitives,” or basic components, that provide an environment for code to be executed and create the magic we experience as online applications.

🔗ThreeFold Compute Primitives

Since we’re getting a bit into the technical weeds here, we’ll just take a brief look at each primitive and what makes it special.

🔗Zmachine

Zmachine is a lightweight virtual machine (VM) implementation that’s been developed in a uniquely efficient way by ThreeFold. VMs are essentially computers within computers, allowing multiple operating systems to run on a single physical system. A common example is running the Windows operating system on a Mac that’s already running MacOS.

In the cloud world, virtual machines are a classic way to manage complex deployments with flexibility and resilience – by packaging different components into different VMs that can be run side by side and moved between different physical servers with ease. While VMs have been replaced in many cases with the new container paradigm, which we’ll cover next, they are still widely used and provide a simpler solution than containers that’s sufficient for many use cases.

🔗Flist

The Flist is a next generation container format which extends the industry standard Docker container concept in an efficient and secure way. It is supported directly at the operating system level on Zero OS, and in its latest version on ThreeFold Grid 3.0, provides a unique advantage over traditional containers via the use of mini virtual machines.

Containers provide greater efficiency and flexibility than VMs, by sharing parts of the host operating system between containers rather than duplicating the full operating system for each component. This, however, means that containers typically come with less of an assurance that different workloads and their data remain isolated from each other. By employing mini VMs, the Flist concept provides better isolation than traditional container systems while retaining the advantages that containers offer over full VMs.

🔗Kubernetes

Containers usually represent small parts of a full solution, which are created, interconnected, and destroyed on an ongoing basis. Kubernetes is the industry standard container orchestration system, and it’s used to manage many of the largest and most complex cloud deployments. Kubernetes solutions are defined through configuration files that specify the types of containers, how updates are handled, and the network architecture both between containers and with the outside world.

Multiple physical servers or VMs are typically used for a Kubernetes “cluster” which provides redundancy and scalability. The ThreeFold Grid provides a lightweight Kubernetes VM that can be used to create clusters according to specific needs. Kubernetes users can, for the most part, simply bring their existing configuration files and recreate their deployments on a Grid based cluster seamlessly.

🔗Wrapping up

Taken together, the compute primitives offer a comprehensive alternative to existing cloud compute providers. Users of Docker, Kubernetes, and virtual machines can bring their deployments to the Grid with a minimal amount of migration effort. This is distinct from other decentralized cloud projects, which either offer a single solution like Kubernetes or a completely new environment that developers must adapt their code to run within.

Better yet, each compute offering on the ThreeFold Grid benefits from the unique efficiency and security features of Zero OS, as well as the unique aspects of the primitives themselves. Thanks again for joining me on this journey into the wonderful world of ThreeFold technology. In the next episode, we’ll cover the last element of the cloud infrastructure triad: network.