Backfill
The process of recomputing embeddings for historical data with a new model and writing them back into the index. The input scale equals the entire historical dataset, and cost grows monotonically with total data volume; it cannot be reduced by adding concurrency. Constrained by throughput limits and cost budgets, it is difficult to complete within a few hours and therefore inevitably overlaps with online queries.
Timeline (last 90 days)
At warehouse scale, re-embedding is a batch workload lasting several days or even longer, and the production index cannot be taken offline before backfilling is complete, so the index is mixed with old and new model vectors throughout the entire window.
The input scale of backfilling equals the full historical data, and its cost increases monotonically with the total data volume and cannot be compressed in total computation by adding concurrency, so in practice backfilling is difficult to complete within a few hours and inevitably overlaps with online queries.
All Facts (2)
At warehouse scale, re-embedding is a batch workload lasting several days or even longer, and the production index cannot be taken offline before backfilling is complete, so the index is mixed with old and new model vectors throughout the entire window.
50%UnverifiedThe input scale of backfilling equals the full historical data, and its cost increases monotonically with the total data volume and cannot be compressed in total computation by adding concurrency, so in practice backfilling is difficult to complete within a few hours and inevitably overlaps with online queries.
50%