MemT-1.0
A foundation model with native memory capability, in which architectural modifications make memory an intrinsic model ability rather than something dependent on external retrieval or long context.
Timeline (last 90 days)
MemT-1.0 treats memory as part of foundation model computation, using architectural modifications to let the foundation model itself accumulate and recall information, rather than relying on retrieval augmentation or ultra-long context.
MemT-1.0 is built on Metis and adopts the Gated DeltaNet-2 architecture.
MemT-1.0 introduces a stable token-level memory update mechanism within each turn, with memory writes occurring at token granularity.
MemT-1.0 designs a Mixture-of-Storage mechanism that partitions different levels of storage units inside the model, allocating and recalling them on demand to expand native memory capacity.
MemT-1.0 adopts a two-stage training pipeline: continued pre-training plus post-training.
MemT-1.0's training pipeline uses 34B tokens of training data.
MemT-1.0 improves success rates on agentic tasks by internalizing retrieved procedural experience.
Native memory approaches face questions of how memory is updated, how it is forgotten, and how to ensure no erroneous information is introduced, which are not yet fully addressed in the paper.
All Facts (8)
MemT-1.0 treats memory as part of foundation model computation, using architectural modifications to let the foundation model itself accumulate and recall information, rather than relying on retrieval augmentation or ultra-long context.
50%UnverifiedMemT-1.0 is built on Metis and adopts the Gated DeltaNet-2 architecture.
50%UnverifiedMemT-1.0 introduces a stable token-level memory update mechanism within each turn, with memory writes occurring at token granularity.
50%UnverifiedMemT-1.0 designs a Mixture-of-Storage mechanism that partitions different levels of storage units inside the model, allocating and recalling them on demand to expand native memory capacity.
50%UnverifiedMemT-1.0 adopts a two-stage training pipeline: continued pre-training plus post-training.
50%UnverifiedMemT-1.0's training pipeline uses 34B tokens of training data.
50%UnverifiedMemT-1.0 improves success rates on agentic tasks by internalizing retrieved procedural experience.
50%UnverifiedNative memory approaches face questions of how memory is updated, how it is forgotten, and how to ensure no erroneous information is introduced, which are not yet fully addressed in the paper.
50%