TTT-MLP
An implementation of the test-time training (TTT) layer that uses an MLP as fast-weight memory, trained online via gradient descent. It relies on sequential chunked updates, which makes parallelization difficult.
Timeline (last 90 days)
Existing TTT implementations such as TTT-MLP rely on sequential, chunked weight updates and run several times slower than chunk-parallel linear recurrence.
At a 4K context, PTTT with two refinement steps evaluates the inner-loop trajectory at matched error 32 times faster than TTT-MLP using chunked approximation.
All Facts (2)
Existing TTT implementations such as TTT-MLP rely on sequential, chunked weight updates and run several times slower than chunk-parallel linear recurrence.
50%UnverifiedAt a 4K context, PTTT with two refinement steps evaluates the inner-loop trajectory at matched error 32 times faster than TTT-MLP using chunked approximation.
50%