Sharded Encrypted Storage Between Friends over Yggdrasil: A New Approach to Decentralized Backup

A Decentralized Storage Concept Worth Watching

As data sovereignty and privacy protection become increasingly important, a project called "Sharded, Encrypted Storage Between Friends over Yggdrasil" has caught the attention of the technical community on Hacker News. Its core idea is straightforward: encrypt your data, split it into multiple shards, distribute them across your friends' devices, and rely on Yggdrasil — an encrypted overlay network — to handle the underlying connectivity.

This design aims to address several pain points in traditional backup solutions — avoiding handing data entirely over to commercial cloud providers while still achieving off-site redundancy. By combining trusted social networks with end-to-end encryption, it maps out a third path between "self-hosted NAS" and "public cloud."

It's worth noting that the original source material is extremely limited (only a title and link, with no comments or detailed documentation available). The following content is an interpretation and extension based on the technical direction implied by the title, provided for reference.

Breaking Down Three Key Technical Terms

Sharded

Sharding refers to splitting a complete dataset into multiple fragments. In storage contexts, this is typically used alongside erasure coding or redundant replication — meaning that even if some shards are lost or a friend's device goes offline, the original data can be reconstructed from the remaining shards. This both improves availability and ensures that no single storage node can see the full picture.

Erasure Coding is the core mathematical tool behind sharded storage and deserves special mention. It works similarly to RAID disk arrays but is more flexible: the original data is split into k data blocks, and an additional m parity blocks are generated, for a total of k+m shards distributed across different nodes. As long as any k shards survive, the original data can be fully reconstructed — allowing up to m nodes to fail simultaneously. For example, a k=3, m=2 configuration means that 2 out of 5 friends can be offline at the same time and the data remains recoverable. Compared to simple multi-copy replication, erasure coding dramatically reduces storage overhead at the same fault-tolerance level — 3-copy replication requires 3× the storage, whereas the erasure coding configuration above requires only 1.67×. Mature distributed storage projects like Tahoe-LAFS and Storj use similar mechanisms and are important references for the reliability design of such systems.

Encrypted

Data is encrypted locally before it ever leaves your device. What gets stored on a friend's machine is nothing but unreadable ciphertext. This means "storing data for a friend" does not equal "being able to read a friend's data" — the mechanism itself eliminates the risk of trust being abused. For peer-to-peer cooperative storage, encryption is the prerequisite for establishing trust boundaries.

Yggdrasil Overlay Network

Yggdrasil is an experimental encrypted IPv6 overlay network that automatically establishes end-to-end encrypted routing between nodes, without requiring complex public IP configuration or centralized server coordination. This fits the needs of friend-based storage perfectly — devices may be scattered across different home broadband connections, behind NAT, and Yggdrasil lets them communicate directly as if they were on the same local network.

Overlay Network is a virtual network layer built on top of the existing internet. Yggdrasil falls into the "encrypted routing" category. It derives a fixed IPv6 address for each node from its public key, and nodes forward packets through a self-organizing tree-based routing protocol with all traffic automatically encrypted. This solves two major obstacles common to home broadband users: NAT traversal (most home routers don't expose a public IP, making P2P direct connections difficult), and dynamic IP address changes (the public-key-derived address is permanent, so a device remains reachable even after switching networks). Compared to similar approaches, Tailscale/WireGuard relies on a centralized coordination server, and ZeroTier also has central nodes, whereas Yggdrasil's routing is fully decentralized — better aligned with the goal of requiring no trusted third party. However, as a result, its routing efficiency and network scale are still in the experimental stage.

Why This Approach Is Appealing

Personal data backup traditionally comes in three flavors: hand it to a public cloud (Dropbox, iCloud, etc.), build your own hardware (NAS, offline drives), or use a decentralized storage network (like IPFS or Storj). Each involves obvious trade-offs.

Public clouds are convenient but put privacy in the hands of service providers, and long-term subscription costs add up. Self-hosted solutions offer strong control but lack off-site disaster recovery — a fire or theft could render local redundancy completely useless. Decentralized networks are forward-thinking in concept, but often depend on token economies or untrusted nodes, making reliability and incentive mechanisms complex.

The elegance of "sharded storage between friends" lies in its use of trust relationships as a foundational resource. You and your friends already trust each other, and you're naturally distributed across different geographic locations and network environments — making you natural, free, off-site, redundant storage nodes for one another. Encryption ensures that even trusted friends can't see the content, and sharding means no single node holds enough to pose a privacy threat.

Real-World Challenges

An elegant concept doesn't mean simple engineering. Systems like this need to tackle several hard problems in practice:

• Availability and uptime: Friends' devices are not 24/7 data centers. If the shards needed to reconstruct data all happen to be offline at the same time, recovery will fail. Careful redundancy design and erasure coding parameters are critical.
• Capacity fairness: Cooperative storage requires some kind of quota or reciprocity mechanism — otherwise it's easy for a "free rider" problem to emerge where some users only consume storage without contributing.
• Key management: The security of encryption ultimately depends on how keys are managed. If the local key is lost, the ciphertext scattered across friends' devices becomes permanently unrecoverable.
• Network maturity: Yggdrasil itself remains an experimental project, and its stability and performance at scale and high bandwidth have yet to be validated.

These challenges mean it's currently better suited as a self-hosting experiment for tech enthusiasts rather than a production-ready solution for general users.

Implications for the Self-Hosting Ecosystem

Setting aside the specific implementation, this project reflects an important trend in the self-hosting and decentralization community in recent years: replacing commercial intermediaries with social relationships. Whether it's decentralized social networks, federated services, or friend-based storage, the underlying motivation is the same — reducing dependence on a single giant, and returning data and control to users and their trusted circles.

For developers and enthusiasts who care about data privacy, the value of projects like this lies not only in whether they can be immediately deployed, but in the framework they offer for rethinking the relationship between storage, trust, and networking. As tools like Yggdrasil and other encrypted overlay networks mature, we may see many more interesting variations of mutual-aid infrastructure built on trusted social networks.

Note: Since this article is based on limited public information, readers interested in this project are encouraged to visit its original repository or the Hacker News discussion page for first-hand technical details and the latest developments.

Background: Key Management

Key management is the weakest link in end-to-end encrypted systems, and is especially critical in friend-based storage scenarios. Common mitigation strategies include: using mnemonic phrases (12/24 English words per the BIP-39 standard) for human-readable backups of master keys; using secret splitting schemes (such as Shamir's Secret Sharing) to split the master key itself into multiple shares held by different trusted contacts, requiring t-of-n shares to reconstruct it — meaning even if the user's own device is completely destroyed, the key can still be recovered with the cooperation of contacts; or encrypting the key and storing it in a dedicated password manager or hardware security key (such as a YubiKey). For self-hosted users, having a clear key backup and disaster recovery procedure is just as important as the data backup itself.