[KongchangAI]
· 4 min read· 2,088 words

Ripple 65B Dense Model Open-Sourced: $32M Training, Takes on DeepSeek-V3

Ripple 65B Dense Model Open-Sourced: $32M Training, Takes on DeepSeek-V3

Chip design company Recursive trained the 65B-parameter dense model Ripple 65B for $32 million and open-sourced it under Apache 2.0.

Recursive, a chip design company founded by the Google Alpha Chip team, has open-sourced Ripple 65B Preview, a 65-billion-parameter dense model pretrained on 12 trillion tokens with a 1-million-token context window under Apache 2.0. It scores 80.52 on Terminal Bench 2.1, beating DeepSeek V3's 80.1, though several Chinese open-source models score in the 88–90 range. Full R&D cost is about $32 million. It uses a dense architecture instead of MoE, an 80-layer design, ~40% MFU, and 1.4 million RL rollouts for post-training. Hands-on tests show it is usable for tool calling and agentic tasks but not an all-rounder.

A lab that designs chips has trained a 65-billion-parameter dense model from scratch, claims the top spot among US open-weight models, and says it cost about $32 million. This is Recursive's Ripple 65B Preview, an open-source model worth taking apart.

Why a chip company is training a language model

Recursive isn't a newcomer to the language model race. Its founding team comes from Google Alpha Chip and worked on reinforcement learning for chip layout. The company's main business is chip design, and its core idea is a closed loop: use AI to design better chips, then use better chips to train better AI.

They need a model to drive their own chip design platform, and they want to keep the entire chain—model training and inference serving—in their own hands. The open-sourced Ripple 65B is the general-purpose version of their internal model; the fully featured chip-manufacturing version remains private. It's unusual for a chip design company to release a coding model, but once you read their technical blog, you'll find it's one of the most detailed public records of model training in a long time.

According to a hands-on video from a Bilibili creator, the model was running on a local DGX workstation immediately after release and was hooked into DeepSeekAd Harness for agentic coding tests.

Back to the core numbers

Ripple 65B release specs: dense architecture, 12 trillion tokens, Apache 2.0

Ripple 65B Preview is a 65-billion-parameter dense model, not a mixture-of-experts model. It was pretrained on 12 trillion tokens and supports a context window of up to 1 million tokens. Twelve trillion tokens is enormous by any standard, but it is indeed less than what many other models actually use in training.

The release is fairly complete: an instruction-tuned model, a BF16 version, an NVFP4 version, and the base model. The base model is especially worth mentioning, because many labs no longer provide one, yet anyone who wants to do their own post-training needs exactly that starting point.

The license is Apache 2.0—commercially usable, fine-tunable, with almost no strange restrictions. If you don't want to run it yourself, the model will be available on multiple cloud platforms, including Together and OpenRouter, with Fireworks AI also planned.

Benchmarks: the caveats behind 80.52

On Terminal Bench 2.1, the officially reported score is 80.52. For reference, DeepSeek V3 has 500 billion parameters and scores 80.1; another open model, MolmoSpark, has 816 billion parameters and scores 76.4. A 65B model beating a 500B model is a headline-grabbing comparison.

But the qualifier matters: Recursive calls it "the most advanced model among US open-weight models." From a global perspective, Chinese open-source models score much higher. DeepSeek Flash 4.1 reports 90.6 on the same benchmark, Xiaomi MiMo 2.6 Pro reports 89.9, and Kimi K3 and JRM 5.3 are both around 88. Roughly ten Chinese open-source models score at least 5 to 10 points higher than Ripple.

Another view is that some of those models were optimized for benchmarks. A fairer comparison might be the closest dense model, Qwen3 32B, also Apache 2.0, which scores 73. Ripple is more than twice its size and scores 7.5 points higher on Terminal Bench.

There's also the issue of evaluation frameworks. Ripple's numbers are self-reported, while independent evaluators running Terminal Bench 2.1 find that models like MiMo 2.6 Pro and Qwen3 32B Max actually score in the 60s, far below vendor-published figures. The evaluation frameworks themselves are changing, so top scores aren't fully comparable.

How the $32 million was spent

This figure is the full R&D cost, not the cost of the final training run. It covers all experiments, data work, evaluation, and post-training. By comparison, MiMo 2.6 mentions that its RL training run cost $2.6 million, but that only counts training itself.

Recursive estimates this is about one-tenth the cost of previous leading open-weight models. This is an estimate based on reported compute transaction prices, so it's worth taking with a grain of salt, but it aligns reasonably well with other information in the industry.

The $32 million is the full R&D cost

Three technical choices worth a closer look

Dense model rather than mixture-of-experts

Almost every other lab is chasing MoE. Ripple goes the other way, arguing that dense models are simpler—simpler to train and simpler to run inference on. They report reaching about 40% MFU in both pretraining and post-training, without using any custom kernels or quantization. For a small team, this is a pragmatic choice. MoE can lower per-token inference cost, but it comes at the price of infrastructure and complexity, and at inference time all expert weights must be loaded into memory.

The architecture is thin and deep: 80 layers, using more attention heads than usual, combined with more sliding-window attention layers, resulting in fewer global parameters. It also uses gated attention, and the global layers do not apply positional encoding. These tricks aren't new in themselves; what's interesting is how they're combined.

The core difference between a dense model and a mixture-of-experts (MoE) model lies in how parameters are activated. In a dense model, all parameters participate in computation during every forward pass, so the inference path is fixed and memory usage is predictable; MoE uses a gating network to activate only some experts for each token, allowing the total parameter count to be much larger than the actual compute, thereby achieving greater capacity at the same compute. Recursive's choice of the dense route means that all 65 billion parameters of Ripple 65B participate in every inference, with no routing overhead and no expert load-balancing issues, significantly reducing the engineering complexity of training and deployment. The cost is that per-token compute is higher than for an MoE model with the same number of activated parameters. This trade-off is especially friendly to small teams, because they don't need to invest additional infrastructure work in expert parallelism, routing stability, and communication optimization.

MFU (Model FLOPs Utilization) is a key metric for measuring training efficiency, representing the ratio of actually achieved floating-point operations to the hardware's theoretical peak. For example, if a GPU has a nominal compute of 1000 TFLOPS and the actual effective compute during training reaches 400 TFLOPS, then MFU is about 40%. This number reflects the combined level of model architecture, parallel strategy, communication overhead, and data pipeline. For a dense model, 40% MFU is quite healthy, because dense architectures don't have MoE's expert routing and load imbalance issues, and the computation pattern is more regular. Recursive emphasizes that this figure was achieved without custom kernels or quantization, meaning the optimization of its training stack comes mainly from engineering tuning rather than special hardware tricks, making it more reproducible.

Model ladder: testing one change at a time

Recursive started with a ladder of small models at around 400 million parameters and worked their way up to large models. The point of the ladder is to cheaply verify whether a given change still works when scaled up. They tried variants of hyper-connections, which worked well at small scale but completely failed above 4 billion parameters, so they abandoned them.

This kind of information disclosure is rare. To give a simple example, small and large models often have different architectures, precisely because what works at small scale may fail at 12B, 26B, or 31B scale.

A model ladder (or scaling ladder) is a systematic scaling-validation method: first train multiple variants at a very small scale (e.g., 400 million parameters), changing only one architectural or training variable at a time to observe its effect; then gradually scale up to intermediate sizes such as 12B, 26B, and 31B to check whether the conclusions from small scale still hold. The advantage is that a single small-model experiment is extremely cheap, allowing you to rule out many invalid directions in a short time and avoid wasting expensive compute on large models. The hyper-connections variant mentioned by Recursive is a typical case: it performed excellently at small scale but completely failed above 4 billion parameters, and going straight to a large model would have caused enormous waste. The essence of the model ladder is to trade low-cost experiments for predictability of scaling laws.

Post-training: 1.4 million RL rollouts

They didn't buy post-training data. Instead, they built their own synthetic data pipeline and environments, continuously adding harder RL tasks as the model improved. They reached 80.52 through 1.4 million reinforcement learning rollouts, and they comment that this may be about 70 times less than what is typically used.

The core story isn't a secret weapon, but a large number of meticulous small-scale experiments, scaling only what survives scaling. They also openly acknowledge standing on the work of organizations like ARC, Prime Intellect, and NVIDIA.

Because those large frontier private model labs

An RL rollout refers to the process in reinforcement learning training where the model generates a complete trajectory for a task environment (possibly including multiple rounds of tool calls, reasoning steps, and a final answer). Each rollout produces training signals used for policy updates. 1.4 million rollouts means the model underwent trial and error and optimization on about 1.4 million task instances, which is on the moderately high side for agentic coding tasks. Recursive says this is about 70 times less than typical usage, implying that many frontier labs use hundreds of millions of rollouts in post-training. The number of rollouts directly determines the compute cost of RL training, and by building its own synthetic data pipeline and difficulty-progressive task environments, they increased the information density of each rollout, thereby reaching the target score with less data.

Hands-on testing: capable, but don't expect it to do everything

The test ran the NVFP4 version on a DGX workstation, with 100% local inference. Once configured correctly, tool calling and agentic tasks all worked properly. There were some initial issues with SGLang, but after fixes it ran smoothly, and generation speed on simple tasks was fast.

One interesting finding: the model doesn't seem to have been pretrained on mass-market datasets, and its output has its own style. In a long-paper writing test, it reviewed and critiqued its own generated paper, identifying several errors and hitting the mark, whereas some domestic models often judge a good paper as full of errors on such tasks. The cost is that the thinking process consumed about 10,000 tokens.

In retrieval-augmented generation tests, answers were concise and to the point, with about 1,700 tokens of thinking. On SVG generation tasks, performance was average—distinctive style but not necessarily better—clearly not a priority optimization area. On Blender-type tasks, it produced basic renders but persisted to completion, consuming millions of tokens, with cost peaking in the 70s.

Let's look at the thinking process

Who is this model for

Ripple isn't the strongest open-source coding model right now; Chinese open-source models generally score higher on benchmarks. But among open-weight models from US labs, it is the strongest small dense model, and it provides both a base model and a public training recipe, with an Apache 2.0 license that is commercially friendly.

If you need a model that runs locally, can be fine-tuned, and comes with complete training details for reference, it's worth watching. Recursive says this is a preview version and that they are scaling toward more frontier levels, so stronger products may follow.

For the open-source ecosystem, the more valuable signal may be this: a company whose main business is chip design, with $32 million and an unusually candid technical report, has proven that a non-frontier lab can train a competitive model from scratch—and has laid the whole process open for everyone to see.

One-line Summary

Chip design company Recursive trained the 65B-parameter dense model Ripple 65B for $32 million and open-sourced it under Apache 2.0.

Paragraph Summary

Recursive, a chip design company founded by the Google Alpha Chip team, has open-sourced Ripple 65B Preview, a 65-billion-parameter dense model pretrained on 12 trillion tokens with a 1-million-token context window under the Apache 2.0 license. It scores 80.52 on Terminal Bench 2.1, beating DeepSeek V3's 80.1, though multiple Chinese open-source models score in the 88–90 range, so the official claim is limited to "strongest US open-weight model." The full R&D cost is about $32 million, covering experiments, data, evaluation, and post-training. Technically, it bucks the trend by choosing a dense architecture over MoE, with an 80-layer thin-and-deep design, about 40% MFU in pretraining and post-training, and post-training via 1.4 million RL rollouts. Hands-on testing shows the model is usable for tool calling and agentic tasks, but not an all-rounder.

#Ripple 65B#Recursive#open-source LLM#dense model#DeepSeek-V3#Terminal Bench#Apache 2.0#MoE
Share:

Related articles