Benchmark
FEB-30
A benchmark for evaluating the memory quality of agents that uses an inversion paradigm to construct deterministic ground truth and tests a system's ability to correctly classify memory operations (creation, update, reinforcement, supersession).
Timeline (last 90 days)
Oct 19
The FEB-30 benchmark proposed in the arXiv paper "Beyond Downstream QA" uses an inversion paradigm to construct deterministic ground truth, and tests show that a representative memory system correctly classifies memory operations (create, update, strengthen, supersede) only 20% of the time.
Unverified50%