Change Data Capture
A technology for tracking database changes and propagating those changes to downstream systems in real time, commonly used in data integration, ETL and event-driven architectures.
Timeline (last 90 days)
In event-sourcing and CDC systems, when the upstream log is truncated, the system inserts gap or truncation marker events so that downstream consumers can perceive the data discontinuity.
Change Data Capture (CDC) is a technique for reading data changes from database transaction logs (such as MySQL binlog, PostgreSQL WAL).
CDC can capture row-level inserts, updates and deletes in near real time without requiring instrumentation changes to business code, and push them downstream as an event stream.
Compared with scheduled batch extraction, CDC can reduce latency from hours to seconds, and it does not put extra pressure on the production database through full-table scans.
CDC brings engineering problems such as schema evolution and splitting of transaction boundaries.
All Facts (5)
In event-sourcing and CDC systems, when the upstream log is truncated, the system inserts gap or truncation marker events so that downstream consumers can perceive the data discontinuity.
50%UnverifiedChange Data Capture (CDC) is a technique for reading data changes from database transaction logs (such as MySQL binlog, PostgreSQL WAL).
50%UnverifiedCDC can capture row-level inserts, updates and deletes in near real time without requiring instrumentation changes to business code, and push them downstream as an event stream.
50%UnverifiedCompared with scheduled batch extraction, CDC can reduce latency from hours to seconds, and it does not put extra pressure on the production database through full-table scans.
50%UnverifiedCDC brings engineering problems such as schema evolution and splitting of transaction boundaries.
50%