etcd: The Cornerstone Data Store for Distributed Systems

What is etcd?

etcd is a distributed, reliable key-value store designed for the most critical data in distributed systems. Written in Go, it currently has 52,481 stars and 10,571 forks on GitHub, with 123 new stars today—a testament to its activity and community recognition.

The core value of etcd lies in providing consistent, highly available configuration sharing and service discovery capabilities for distributed systems. Whether it's storing Kubernetes cluster state or handling service registration and discovery, etcd plays the role of a "data cornerstone."

Core Features

Strong Consistency and High Availability

etcd achieves strong consistency based on the Raft consensus algorithm, ensuring that the cluster can still read and write normally as long as a majority of nodes are alive. This means that even if some nodes fail, the system won't lose critical data—making it ideal for scenarios with extremely high reliability requirements.

Raft is a consensus algorithm designed for understandability, managing the replicated log through three sub-problems: leader election, log replication, and safety constraints. At any given moment, at most one Leader in the cluster accepts write requests; the Leader replicates operations as log entries to a majority of nodes and only returns success after they are committed. Therefore, writes require confirmation from more than half the nodes, which explains why etcd recommends deploying an odd number of nodes (typically 3 or 5). Raft and Paxos share similar goals but Raft is easier to implement in engineering, and etcd's raft library has been reused by numerous projects. It's important to distinguish that Raft guarantees strong consistency (linearizable reads and writes), while high availability depends on majority survival: when the cluster loses its majority, etcd rejects writes to protect consistency—a classic CP choice in the CAP trade-off.

Simple Key-Value Interface

etcd provides HTTP/gRPC interfaces supporting CRUD operations on key-values, watch, and lease mechanisms. The watch mechanism allows clients to perceive data changes in real time, forming the foundation for service discovery and hot configuration updates.

A lease is a TTL-based authorization mechanism in etcd: a client requests a lease and binds several keys to it, and must periodically renew it (keepalive). Once the lease expires or the client disconnects, the bound keys are automatically deleted. This mechanism is naturally suited for service registration—instances write keys with leases when they come online, and keys are automatically cleaned up after a crash, with no manual intervention needed. Watch is implemented based on etcd's MVCC (Multi-Version Concurrency Control): each modification generates a new revision number, and clients can specify a starting revision for watching, thereby supporting historical event replay and no event loss after reconnection. The combination of watch and leases constitutes the foundational capability for service discovery, hot configuration updates, and distributed coordination.

Security and Operations-Friendly

etcd supports TLS-encrypted communication and role-based access control (RBAC), while also providing comprehensive monitoring metrics and backup/recovery tools for convenient production operations.

Typical Use Cases

• Kubernetes cluster state storage: Kubernetes uses etcd to store the state of all cluster objects—the most well-known application of etcd.
• Service registration and discovery: Using leases and watch mechanisms, service instances can automatically register and be removed upon failure.
• Distributed locks and leader election: Distributed coordination implemented through transactions and leases.
• Configuration management: Centralized configuration management with real-time change propagation.

Distributed locks are typically implemented using etcd's transactions (Txn) and leases: a client checks in a transaction whether a key doesn't exist (comparing revision to 0), and if so, writes and binds a lease, atomically acquiring the lock; the lock holder must continuously renew, and the lock is automatically released when the lease expires after a crash. Leader election follows a similar approach—multiple candidates compete to write to the same key, the winner becomes the leader, and leader changes are detected via watch. Additionally, etcd's transactions support CAS (compare-and-swap) semantics based on revision numbers, which is key to implementing optimistic concurrency control—one of the reasons it can support high-concurrency coordination scenarios like Kubernetes.

Community and Ecosystem

etcd is a CNCF graduated project with a mature community and broad ecosystem integration. Its Go implementation ensures convenient cross-platform deployment, and the 52,481 stars and 10,571 forks confirm its standing in the distributed systems field. For teams building reliable distributed systems, etcd is a choice validated by large-scale production use.

Conclusion

With strong consistency, high availability, and a simple API, etcd has become one of the de facto standards for critical data storage in distributed systems. Whether for container orchestration, service discovery, or distributed coordination, etcd provides solid underlying support. If you're designing a distributed system, etcd is worth exploring in depth.