Janas
An open-source project that streams expert weights from a mixture-of-experts (MoE) model off an SSD, enabling large language models to run on consumer-grade hardware whose memory is insufficient to hold the full model.
Timeline (last 90 days)
Genus is an inference engine written in pure C, and the author is Janas.
The author Janas explicitly stated that Genus is just an experiment, not yet a product.
Janas uses O_DIRECT to read expert weights on demand and tracks hot experts to keep them in an in-memory cache, thereby avoiding system-level page-fault storms.
Janas supports speculative decoding: a small draft model guesses the next few tokens and the large model verifies them in one pass. On Qwen3-Next-80B this raises write speed from 24 to 37 tok/s, with the text content completely identical.
On the test machine, Qwen3-Next-80B reads a 1024-token prompt at 62 tok/s and writes at 24 tok/s without the draft enabled; Qwen3-Coder-Next, also 48.4GB, reads at 66 and writes at 22.
The author claims Janas beats llama.cpp on all 13 compared models, under the same test conditions of the same laptop and Q4_K_M quantization.
Janas is used on Linux x86-64 (Ubuntu 22.04, Debian 12 and above) by downloading the tar.gz package, extracting it and running ./janas-chat; Windows 10 (1903+) and 11 have a zip version, and the author reports write speeds on Windows 11 on par with Linux.
Janas is a language model inference engine written in C, by Maurizio Cammalleri.
Janas runs the 48.4GB Qwen3-Next-80B on a 32GB-memory laptop at 37 tok/s; on the same machine and same model llama.cpp reaches only 2.8 tok/s.
All Janas performance data comes from the project's public benchmarks, README and Hacker News discussion thread, and is the author's own testing rather than independent third-party verification.
5 more timeline events
All Facts (15)
Genus is an inference engine written in pure C, and the author is Janas.
50%UnverifiedThe author Janas explicitly stated that Genus is just an experiment, not yet a product.
50%UnverifiedJanas uses O_DIRECT to read expert weights on demand and tracks hot experts to keep them in an in-memory cache, thereby avoiding system-level page-fault storms.
50%UnverifiedJanas supports speculative decoding: a small draft model guesses the next few tokens and the large model verifies them in one pass. On Qwen3-Next-80B this raises write speed from 24 to 37 tok/s, with the text content completely identical.
50%UnverifiedOn the test machine, Qwen3-Next-80B reads a 1024-token prompt at 62 tok/s and writes at 24 tok/s without the draft enabled; Qwen3-Coder-Next, also 48.4GB, reads at 66 and writes at 22.
50%UnverifiedThe author claims Janas beats llama.cpp on all 13 compared models, under the same test conditions of the same laptop and Q4_K_M quantization.
50%UnverifiedJanas is used on Linux x86-64 (Ubuntu 22.04, Debian 12 and above) by downloading the tar.gz package, extracting it and running ./janas-chat; Windows 10 (1903+) and 11 have a zip version, and the author reports write speeds on Windows 11 on par with Linux.
50%UnverifiedJanas is a language model inference engine written in C, by Maurizio Cammalleri.
50%UnverifiedJanas runs the 48.4GB Qwen3-Next-80B on a 32GB-memory laptop at 37 tok/s; on the same machine and same model llama.cpp reaches only 2.8 tok/s.
50%UnverifiedAll Janas performance data comes from the project's public benchmarks, README and Hacker News discussion thread, and is the author's own testing rather than independent third-party verification.
50%UnverifiedJanas uses the integrated GPU to process the prompt, while in the token generation phase it streams expert weights from the SSD; JANAS_GPU=0 can be used to force pure CPU mode.
50%UnverifiedJanas-style streaming loading schemes depend heavily on NVMe, and on SATA drives the random read latency is so high that inference speed drops to an unusable level.
50%UnverifiedThe core claim of the Janas project is to run large language models that exceed memory capacity by streaming experts from an SSD.
50%UnverifiedJanas's engineering value lies in decoupling "what can be used" from "what can be stored": the SSD is responsible for storage, memory is responsible for use, and the two are bridged in between through streaming scheduling.
50%UnverifiedThe Janas project recently appeared on Hacker News.
50%