Optimistic Concurrency Control
A concurrency control mechanism that assumes transaction conflicts are infrequent, allowing operations to execute without acquiring locks. Conflicts are detected at commit time through atomic validation. The approach is widely used in distributed systems and database transaction management.
Timeline (last 90 days)
Lakehouse solutions mostly adopt optimistic concurrency control together with atomic commits at the metadata layer.
Optimistic concurrency control checks only at commit time whether there is a conflict with other transactions, and if there is a conflict, it rolls back and retries; throughput is higher in low-conflict scenarios, but when writing concurrently to the same partition or the same table, the number of retries rises significantly and can even lead to a higher transaction failure rate.
In high-contention financial hotspot scenarios, optimistic concurrency control causes most transactions to fail the version check, triggering a retry storm.
High-reliability ledgers use optimistic concurrency control (OCC) together with immutable append constraints, or use pessimistic row-level locking (SELECT ... FOR UPDATE) on account pointer records.
The "last-write trap" described in the article is the Lost Update problem in distributed systems; one standard solution is optimistic concurrency control, which operates on the same principle as a database CAS operation.
Before committing to storage, the target record, permitted change types, and current record version number should be validated; if they are inconsistent with the proposal, the current record should be retained and the conflict routed to a human for handling.
Iceberg and Delta Lake use optimistic concurrency control mechanisms, where write operations commit transactions by atomically updating the pointer to the current metadata file.
When two catalogs each permit writes, they may independently advance their respective maintained pointers, causing one commit to overwrite the other and resulting in data loss or state inconsistency.
All Facts (8)
Lakehouse solutions mostly adopt optimistic concurrency control together with atomic commits at the metadata layer.
50%UnverifiedOptimistic concurrency control checks only at commit time whether there is a conflict with other transactions, and if there is a conflict, it rolls back and retries; throughput is higher in low-conflict scenarios, but when writing concurrently to the same partition or the same table, the number of retries rises significantly and can even lead to a higher transaction failure rate.
50%UnverifiedIn high-contention financial hotspot scenarios, optimistic concurrency control causes most transactions to fail the version check, triggering a retry storm.
50%UnverifiedHigh-reliability ledgers use optimistic concurrency control (OCC) together with immutable append constraints, or use pessimistic row-level locking (SELECT ... FOR UPDATE) on account pointer records.
50%UnverifiedThe "last-write trap" described in the article is the Lost Update problem in distributed systems; one standard solution is optimistic concurrency control, which operates on the same principle as a database CAS operation.
50%UnverifiedBefore committing to storage, the target record, permitted change types, and current record version number should be validated; if they are inconsistent with the proposal, the current record should be retained and the conflict routed to a human for handling.
50%UnverifiedIceberg and Delta Lake use optimistic concurrency control mechanisms, where write operations commit transactions by atomically updating the pointer to the current metadata file.
50%UnverifiedWhen two catalogs each permit writes, they may independently advance their respective maintained pointers, causing one commit to overwrite the other and resulting in data loss or state inconsistency.
50%