deltafin
An open-source project that enables the distributed storage of very large model weights across multiple SSDs. Through streaming-based loading, it supports large model inference on consumer-grade devices with limited memory.
Timeline (last 90 days)
A developer ran the 2.8 trillion-parameter Kimi K3 model on a single MacBook Pro at approximately 1 token/second.
deltafin distributes model weights across four SSDs and performs inference via streaming loading.
The sparse activation characteristic of MoE means that streaming inference only needs to read the weights of activated experts, which is the key prerequisite for the feasibility of deltafin.
deltafin's workload is predominantly sequential reads; the direct impact of pure read scenarios on SSD TBW is relatively limited, but high-intensity continuous reads raise operating temperatures and may cause performance throttling.
All Facts (4)
deltafin distributes model weights across four SSDs and performs inference via streaming loading.
50%UnverifiedThe sparse activation characteristic of MoE means that streaming inference only needs to read the weights of activated experts, which is the key prerequisite for the feasibility of deltafin.
50%Unverifieddeltafin's workload is predominantly sequential reads; the direct impact of pure read scenarios on SSD TBW is relatively limited, but high-intensity continuous reads raise operating temperatures and may cause performance throttling.
50%UnverifiedA developer ran the 2.8 trillion-parameter Kimi K3 model on a single MacBook Pro at approximately 1 token/second.
50%