TensorFold
A local large-model inference engine that supports efficient execution of mixed-precision quantized models.
Timeline (last 90 days)
Early-stage inference engines of this kind usually have uneven backend maturity: the CUDA toolchain ecosystem is the most complete, while the implementation depth for Metal and Vulkan often depends on the maintainer's investment.
TensorFold separates backend abstraction from operator implementation, keeping the upper-level inference scheduling logic unified while replacing kernels per platform at the lower level.
The maintenance burden of a multi-backend inference engine multiplies, as operator coverage, numerical consistency and performance tuning for each backend all need to be tracked independently.
TensorFold has moved past the personal-experiment stage and entered the range of open-source projects with real attention.
TensorFold is an open-source cross-platform LLM inference engine written in Zig and targeting Metal, CUDA and Vulkan.
When serving Qwen and Nemotron models with up to 125B parameters on a single DGX Spark, Hibana completes up to 3.4 times more agent tasks per hour and achieves up to 3.9 times higher decoding throughput than vLLM, NVIDIA Flare, llama.cpp and TensorFold.
TensorFold's build runs a hybrid 2/3-bit model on an 80-core M5 Ultra chip at 70-105 tokens/s written; the MLX route achieves 26-29 tokens/s written on 64 cores.
Magnitude has accumulated 1,741 stars on GitHub, with 130 new stars added in a single day and a fork count of 134.
All Facts (7)
TensorFold separates backend abstraction from operator implementation, keeping the upper-level inference scheduling logic unified while replacing kernels per platform at the lower level.
50%UnverifiedTensorFold has moved past the personal-experiment stage and entered the range of open-source projects with real attention.
50%UnverifiedEarly-stage inference engines of this kind usually have uneven backend maturity: the CUDA toolchain ecosystem is the most complete, while the implementation depth for Metal and Vulkan often depends on the maintainer's investment.
50%UnverifiedTensorFold is an open-source cross-platform LLM inference engine written in Zig and targeting Metal, CUDA and Vulkan.
50%UnverifiedThe maintenance burden of a multi-backend inference engine multiplies, as operator coverage, numerical consistency and performance tuning for each backend all need to be tracked independently.
50%UnverifiedWhen serving Qwen and Nemotron models with up to 125B parameters on a single DGX Spark, Hibana completes up to 3.4 times more agent tasks per hour and achieves up to 3.9 times higher decoding throughput than vLLM, NVIDIA Flare, llama.cpp and TensorFold.
50%UnverifiedTensorFold's build runs a hybrid 2/3-bit model on an 80-core M5 Ultra chip at 70-105 tokens/s written; the MLX route achieves 26-29 tokens/s written on 64 cores.
50%