Object Storage
A cloud storage architecture paradigm that stores data objects in a flat namespace. Each object contains data, metadata, and a unique identifier. The approach is well-suited for large, immutable data, offers low cost and elastic capacity, and is represented by products such as Amazon S3 and Google Cloud Storage.
Core Facts
Compared to block storage, the unit storage cost of object storage is typically one order of magnitude lower
The access latency of object storage is typically in the range of tens to hundreds of milliseconds, far higher than local SSD (microsecond-level) or memory (nanosecond-level).
Timeline (last 90 days)
Object storage (such as S3-compatible interfaces) has become the mainstream carrier for large-scale AI datasets due to its near-unlimited scalability.
Updates to refs require strong consistency guarantees, whereas most object storage only provides eventual consistency, requiring additional metadata services or conditional write mechanisms to avoid conflicts.
In an object storage scenario, issuing a range request to object storage only when the actual content of a specific object needs to be read is the key prerequisite for achieving acceptable latency through this decoupled design.
Git can run on object storage if one is willing to regenerate packfiles.
Object storage lacks the directory semantics, file locks, and low-latency random access capabilities of a traditional file system.
HydraDB is built directly on top of object storage rather than relying on traditional local disks or self-managed storage layers.
Building on top of object storage reduces operational complexity, eliminating the need to manage storage nodes, handle disk failures, or manually scale capacity.
Major cloud service providers generally promise 99.999999999% (eleven nines) data durability for object storage
Object storage manages data as objects, where each object contains the data itself, metadata, and a globally unique identifier
Compared to block storage, the unit storage cost of object storage is typically one order of magnitude lower
12 more timeline events
All Facts (20)
Compared to block storage, the unit storage cost of object storage is typically one order of magnitude lower
75%VerifiedThe access latency of object storage is typically in the range of tens to hundreds of milliseconds, far higher than local SSD (microsecond-level) or memory (nanosecond-level).
65%UnverifiedObject storage treats data as independent "objects," where each object contains the data itself, metadata, and a unique identifier, stored in a flat namespace, which is fundamentally different from traditional file storage (hierarchical directory structure) and block storage (fixed-size blocks mapped to disk sectors).
60%UnverifiedObject storage (such as S3-compatible interfaces) has become the mainstream carrier for large-scale AI datasets due to its near-unlimited scalability.
50%UnverifiedIn an object storage scenario, issuing a range request to object storage only when the actual content of a specific object needs to be read is the key prerequisite for achieving acceptable latency through this decoupled design.
50%UnverifiedGit can run on object storage if one is willing to regenerate packfiles.
50%UnverifiedObject storage lacks the directory semantics, file locks, and low-latency random access capabilities of a traditional file system.
50%UnverifiedUpdates to refs require strong consistency guarantees, whereas most object storage only provides eventual consistency, requiring additional metadata services or conditional write mechanisms to avoid conflicts.
50%UnverifiedBuilding on top of object storage reduces operational complexity, eliminating the need to manage storage nodes, handle disk failures, or manually scale capacity.
50%UnverifiedHydraDB is built directly on top of object storage rather than relying on traditional local disks or self-managed storage layers.
50%UnverifiedMajor cloud service providers generally promise 99.999999999% (eleven nines) data durability for object storage
50%UnverifiedObject storage manages data as objects, where each object contains the data itself, metadata, and a globally unique identifier
50%UnverifiedObject storage API requests (PUT/GET/LIST) are themselves billed, and when accessing high-frequency small objects, request costs can far exceed storage costs
50%UnverifiedAggregating small objects into large objects can reduce request overhead for object storage
50%Unverified将数据层外置到对象存储的策略本质上是将复杂性从应用运维侧转移到专业的存储服务侧
50%Unverified对象存储采用扁平化的命名空间，通过 RESTful HTTP API 进行数据存取，将数据作为包含数据、元数据和唯一标识符的对象来管理
50%Unverified将数据层外置到对象存储可实现应用无状态化，从而支持容器化部署（如 Docker Compose）和 Kubernetes 编排
50%UnverifiedPicoMQ 将持久化消息流直接构建在对象存储之上，并通过 HTTP 协议提供访问
50%UnverifiedS3 等对象存储仅提供简单的 PUT/GET 原语，不支持原子性的 compare-and-swap 操作或事务日志
50%Unverified将数据层外置到对象存储的策略本质上是将复杂性从应用运维侧转移到专业的存储服务侧，是实现云原生应用无状态化的关键手段之一
50%