Nemotron Labs Dory 1.0
A 4B-parameter recurrent-architecture language model released by NVIDIA. It employs an input block, a shared recurrent block and an output block, supports dynamic adjustment of the number of loop iterations at inference time, and focuses on agentic coding tasks.
Timeline (last 90 days)
NVIDIA Nemotron Labs Dory 1.0 is a 4B-parameter model that went live on Hugging Face on October 16.
Dory scores 61.8 on SWE-bench Verified (4 loops, NVIDIA's own testing), while Qwen3.5-9B scores 61.
As of October 18, officially supported llama.cpp, vLLM and Transformers have not merged support for Dory, or it is missing.
Dory uses a recurrent architecture consisting of an input block (8 layers), a shared recurrent block (20 layers, repeatable 1 to 4 times) and an output block (8 layers); the input is re-injected on each loop, and the number of loops can be specified at startup without retraining.
At 4 loops, Dory computes 8 + 80 + 8 = 96 layers per token.
NVIDIA recommends 2 loops as the default and says most of the accuracy gain comes from the second loop.
The Dory model specifications are: a 131,072-token vocabulary, hidden dimension 2560, 16 query heads, 4 KV heads, head dimension 256, about 6T pretraining tokens, English-only support, and training covering 43 programming languages.
Dory uses the OpenMDW License v1.1, and the model card states it can be used for commercial or non-commercial purposes.
Dory's score on the AA-Omniscience benchmark barely changes from 1 loop to 4 loops (10.03, 10.08, 10.03), and it trails Qwen3.5-4B on both GPQA Diamond and HLE.
Dory leads Qwen3.5-4B by more than 20 points on IFBench (79 vs 53.83, 4 loops).
4 more timeline events
All Facts (14)
NVIDIA says last_loop mode's accuracy is close to that of per-loop mode.
50%UnverifiedDory scores 61.8 on SWE-bench Verified (4 loops, NVIDIA's own testing), while Qwen3.5-9B scores 61.
50%UnverifiedAs of October 18, officially supported llama.cpp, vLLM and Transformers have not merged support for Dory, or it is missing.
50%UnverifiedDory uses a recurrent architecture consisting of an input block (8 layers), a shared recurrent block (20 layers, repeatable 1 to 4 times) and an output block (8 layers); the input is re-injected on each loop, and the number of loops can be specified at startup without retraining.
50%UnverifiedAt 4 loops, Dory computes 8 + 80 + 8 = 96 layers per token.
50%UnverifiedNVIDIA recommends 2 loops as the default and says most of the accuracy gain comes from the second loop.
50%UnverifiedThe Dory model specifications are: a 131,072-token vocabulary, hidden dimension 2560, 16 query heads, 4 KV heads, head dimension 256, about 6T pretraining tokens, English-only support, and training covering 43 programming languages.
50%UnverifiedDory uses the OpenMDW License v1.1, and the model card states it can be used for commercial or non-commercial purposes.
50%UnverifiedDory's score on the AA-Omniscience benchmark barely changes from 1 loop to 4 loops (10.03, 10.08, 10.03), and it trails Qwen3.5-4B on both GPQA Diamond and HLE.
50%UnverifiedDory leads Qwen3.5-4B by more than 20 points on IFBench (79 vs 53.83, 4 loops).
50%UnverifiedDory's GGUF weights (Q5_K_M) are 3.67 GB, BF16 is 7.95 GB, and NVFP4 is 3.20 GB.
50%UnverifiedIn the default per_loop mode, Dory's KV cache multiplies with the number of loops: about 14 GiB for a 256K context at 2 loops and about 24 GiB at 4 loops; switching to last_loop mode reduces this to about 9 GiB (256K).
50%Unverifiedllama.cpp's support for Dory is in PR #30306, currently in draft status and requiring two approvals, and the PR description acknowledges that Metal is untested and long-context qualification is incomplete.
50%UnverifiedNVIDIA Nemotron Labs Dory 1.0 is a 4B-parameter model that went live on Hugging Face on October 16.
50%