Storage-Compute Separation
An architectural pattern in which the data persistence layer and the compute layer are deployed independently and scaled independently. It can improve elasticity and resource utilization, but introduces complexity in metadata coordination and consistency.
Timeline (last 90 days)
Storage-compute separation means deploying the data persistence layer and the compute layer independently and scaling them independently; in a search system, index building, merging, and query serving can run on different node pools while sharing the same S3 data.
Storage-compute separation is not a universal answer but an explicit trade-off: trading latency and complexity for cost, elasticity, and durability.
Traditional databases bind storage and compute on the same node, requiring both to be upgraded simultaneously when scaling.
Decoupling storage from compute allows teams to independently select a cheaper storage tier for cold data without being forced to upgrade expensive compute instances when expanding storage.
Storage-compute separation architecture achieves second-level elastic scaling of compute resources by decoupling the storage layer from the compute layer, allowing storage to scale independently without data migration.
Storage-compute separation architectures are highly dependent on caching, and insufficient cache warm-up during service restarts, scale-out, or sudden changes in access patterns can cause latency spikes in the short term.
In a storage-compute separation architecture, data access requires crossing network boundaries, and even within the same availability zone, round-trip network latency is one to two orders of magnitude higher than that of local NVMe SSDs.
All Facts (7)
Storage-compute separation means deploying the data persistence layer and the compute layer independently and scaling them independently; in a search system, index building, merging, and query serving can run on different node pools while sharing the same S3 data.
50%UnverifiedStorage-compute separation is not a universal answer but an explicit trade-off: trading latency and complexity for cost, elasticity, and durability.
50%UnverifiedTraditional databases bind storage and compute on the same node, requiring both to be upgraded simultaneously when scaling.
50%UnverifiedDecoupling storage from compute allows teams to independently select a cheaper storage tier for cold data without being forced to upgrade expensive compute instances when expanding storage.
50%UnverifiedStorage-compute separation architecture achieves second-level elastic scaling of compute resources by decoupling the storage layer from the compute layer, allowing storage to scale independently without data migration.
50%UnverifiedStorage-compute separation architectures are highly dependent on caching, and insufficient cache warm-up during service restarts, scale-out, or sudden changes in access patterns can cause latency spikes in the short term.
50%UnverifiedIn a storage-compute separation architecture, data access requires crossing network boundaries, and even within the same availability zone, round-trip network latency is one to two orders of magnitude higher than that of local NVMe SSDs.
50%