Agens Volundr 32B
A 32-billion-parameter hybrid-architecture large language model developed by the Blockway team. It adopts a 72-layer structure in which only 18 layers retain KV caches, and integrates mechanisms including linear attention (KDA) and block-compressed sparse attention (BCSA), supporting a context window of up to 262K tokens.
Timeline (last 90 days)
Agens Volundr 32B is a dense architecture with approximately 32B parameters and 72 layers in total, of which only 18 layers retain KV cache.
The model's layer composition is 54 layers of KDA (Kimi Delta Attention) linear attention, 17 layers of BCSA compressed sparse attention, and 1 full attention layer (layer 72).
Engram is a hash-based n-gram memory stored in host memory and attached to 2 of the 72 layers; mHC replaces the traditional single residual stream with 4 residual streams.
Agens Volundr 32B has a context window of 262K.
In BF16 configuration (two 48GB GPUs), decoding speed ranges from 25.1 tok/s at 1K to 23.9 tok/s at 128K, with almost no degradation.
Benchmarked against Qwen3.8-27B, Agens Volundr 32B leads on LiveCodeBench v6 (+4.2), HumanEval (+4.3), AIME 2025 (+2.9), and MATH-500 (+1.6).
On agent tasks, Agens Volundr 32B lags behind: tau2-bench scores 74.2 (competitor 79–80), and SWE-bench Verified (50-question subset) is only 44 (competitor 58–64).
The full v1 will continue pretraining to approximately 10 billion tokens and will incorporate training on long agent sessions to narrow the gap on agent tasks.
The model currently must use the team's own sglang build; the original sglang and vLLM cannot load it, and GGUF/llama.cpp support is still planned.
The model is open-sourced under the Apache-2.0 license; BF16, INT4 weights, and the DFlash2 drafter have been uploaded to Hugging Face, with Docker images provided for sm89 and sm90.
All Facts (9)
Agens Volundr 32B is a dense architecture with approximately 32B parameters and 72 layers in total, of which only 18 layers retain KV cache.
50%UnverifiedThe model's layer composition is 54 layers of KDA (Kimi Delta Attention) linear attention, 17 layers of BCSA compressed sparse attention, and 1 full attention layer (layer 72).
50%UnverifiedEngram is a hash-based n-gram memory stored in host memory and attached to 2 of the 72 layers; mHC replaces the traditional single residual stream with 4 residual streams.
50%UnverifiedAgens Volundr 32B has a context window of 262K.
50%UnverifiedIn BF16 configuration (two 48GB GPUs), decoding speed ranges from 25.1 tok/s at 1K to 23.9 tok/s at 128K, with almost no degradation.
50%UnverifiedBenchmarked against Qwen3.8-27B, Agens Volundr 32B leads on LiveCodeBench v6 (+4.2), HumanEval (+4.3), AIME 2025 (+2.9), and MATH-500 (+1.6).
50%UnverifiedOn agent tasks, Agens Volundr 32B lags behind: tau2-bench scores 74.2 (competitor 79–80), and SWE-bench Verified (50-question subset) is only 44 (competitor 58–64).
50%UnverifiedThe full v1 will continue pretraining to approximately 10 billion tokens and will incorporate training on long agent sessions to narrow the gap on agent tasks.
50%UnverifiedThe model is open-sourced under the Apache-2.0 license; BF16, INT4 weights, and the DFlash2 drafter have been uploaded to Hugging Face, with Docker images provided for sm89 and sm90.
50%