Step 5 Preview In-Depth: A 600B Open-Source MoE Model Aimed at Long-Running Agents

StepFun has released Step 5 Preview on Product Hunt, an open-source Mixture-of-Experts (MoE) model with 600B total parameters. Its core selling point isn't sheer parameter scale, but rather specialized training for coding, financial research, and long-running agents, along with a 1M-token ultra-long context and out-of-the-box support for mainstream inference frameworks. At a time when open-source model competition is increasingly homogeneous, this positioning targets a real pain point: agents tend to "drift off" or forget their objectives during long-duration tasks.

Model Specs: Balancing 600B Total Parameters with 27B Activated Parameters

Step 5 Preview uses an MoE architecture with 600B total parameters, but only 27B parameters are activated per token. This design keeps the model's capability ceiling high while keeping inference costs relatively acceptable—activated parameters directly determine the cost of a single forward pass. For agent tasks that need to run continuously over long periods, inference costs accumulate linearly with runtime, making sparse activation a pragmatic choice.

The 1M-token context window means the model can process extremely long codebases, multi-turn tool-call records, or lengthy financial documents in one pass, without frequent truncation or reliance on external memory mechanisms. For long-running agents, context length directly affects task coherence.

For deployment, the official release explicitly supports the three mainstream frameworks vLLM, SGLang, and Transformers, lowering the barrier to entry for developers. Open-source weights mean enterprises can deploy in private environments, meeting data compliance requirements in sensitive industries such as finance.

Mixture-of-Experts (MoE) architecture works by splitting the model's feed-forward network layers into multiple parallel "expert" sub-networks, then using a lightweight router network to dynamically select and activate only a few experts for each input token. Unlike dense models where every token passes through all parameters, MoE decouples "total parameter count" from "per-pass computation": total parameters determine the model's capacity ceiling and knowledge reserve, while activated parameters determine the floating-point operations per token. This sparse activation lets models scale substantially in parameter count while inference costs grow only with activated parameters. The trade-offs are that memory still needs to hold all expert weights, and imbalanced routing may leave some experts under-trained, so MoE training typically requires auxiliary mechanisms like load-balancing loss. Step 5 Preview's 600B total parameters correspond to roughly 27B activated parameters—an activation ratio under 5%, placing it in the high-sparsity configuration.

Why Focus on "Long-Running Agents"?

"Long-running agents" are currently a key bottleneck for putting agents into practice. Most demo-grade agents can only complete short tasks within minutes; once a task spans hours or even days, models tend to exhibit goal drift, repetitive operations, context forgetting, and other issues. By making this scenario a training priority, Step 5 Preview reflects the vendor's assessment of real-world pain points: an agent's value lies not in impressive single-turn Q&A, but in whether it can reliably complete multi-step, long-cycle complex workflows.

Coding scenarios are naturally suited to long-running agents—modifying large codebases, multi-file refactoring, and continuous integration debugging all require the model to maintain an understanding of project state over long time spans. The same applies to financial research: from data retrieval and metric computation to report generation, it often involves dozens of tool calls and repeated verification. The 600B total capacity provides the knowledge reserve for these complex tasks, while the MoE structure keeps actual inference from becoming prohibitively expensive.

Goal drift and context forgetting are two typical failure modes of long-running agents. Goal drift means that after dozens of rounds of tool calls and environmental feedback, the agent gradually deviates from the original task objective and instead executes sub-tasks unrelated to or suboptimal for the original intent; context forgetting means that key constraints (such as user preferences, confirmed facts, and ruled-out options) are diluted or overwritten in a long context, leading to duplicated work or self-contradictory decisions. The common root of both lies in the fact that the model's attention mechanism lacks a persistent anchor for "task state," and each step's locally optimal decision may not point to the global optimum. Common industry mitigations include explicitly maintaining a task list, periodically compressing state into summaries, introducing external memory storage, and setting up checkpoint rollback mechanisms. Making long-running operation a training objective means specifically strengthening the model's ability to maintain long-term goals through data construction and reward design, rather than optimizing only per-step correctness.

The Practical Significance of Open-Source Weights and Deployment Support

Open-source weights are the core differentiator that sets Step 5 Preview apart from closed-source APIs. For both coding and financial research scenarios, data sensitivity is an unavoidable issue: enterprise codebases and financial data are typically not permitted to be uploaded to third-party APIs. Open-source weights, combined with high-performance inference frameworks like vLLM and SGLang, allow teams to deploy on their own infrastructure while leveraging optimizations such as PagedAttention and continuous batching to improve throughput.

Native Transformers support makes rapid validation and fine-tuning convenient. Developers can first use Transformers for prototyping, then switch to vLLM or SGLang for production deployment. This tiered support strategy reduces the migration cost from experimentation to launch.

PagedAttention is a KV Cache memory management technique introduced by vLLM. Drawing on the paging concept from operating system virtual memory, it divides each sequence's key-value cache into fixed-size blocks allocated on demand, eliminating memory waste caused by pre-allocating the maximum length. In long-context and high-concurrency scenarios, this mechanism can significantly improve memory utilization and support larger batch sizes. Continuous batching allows new requests to be dynamically inserted into a batch and completed requests to be removed during inference, without waiting for the entire batch to finish—especially critical for agent scenarios where request lengths vary widely. SGLang further optimizes at the scheduling and front-end language levels, making it suitable for complex multi-turn call structures. Understanding the principles behind these framework optimizations helps teams judge whether a 1M-token context can deliver acceptable latency and throughput in real deployments, rather than being merely a nominal specification.

Competitive Landscape and Open Questions

In the open-source MoE race, Step 5 Preview faces competition from several peers of similar scale. Configurations with 600B total parameters and tens of billions of activated parameters have become common specs among leading open-source models. Step 5 Preview's differentiation lies in its scenario focus—explicitly making long-running agents a training objective rather than pursuing all-around leadership on general benchmarks.

However, the product page discloses limited information: the composition of training data, the specific evaluation methodology for long-running agents, and comparative data against similar models on long-cycle tasks are all undisclosed. Community feedback of 71 upvotes and 6 comments also indicates the model is still in an early exposure stage. How capable it actually is at long-running agent tasks still needs to be verified by the developer community on real tasks. Whether the model weights are fully open for commercial use and what the license terms are also deserve attention.

For teams building coding assistants or financial analysis agents, Step 5 Preview offers a new option worth evaluating. It's advisable to first run small-scale tests on your own business's long-cycle tasks, focusing on the model's goal retention and context utilization after dozens of rounds of tool calls.

One-line Summary

StepFun releases an open-source MoE model with 600B total parameters and 27B activated, targeting long-running agents and coding/financial scenarios.

Paragraph Summary

StepFun released Step 5 Preview on Product Hunt, an open-source Mixture-of-Experts model with 600B total parameters that activates only 27B per token. Its differentiation isn't parameter scale but explicit specialized training for coding, financial research, and long-running agent scenarios, paired with a 1M-token ultra-long context. Sparse activation keeps inference costs manageable for long continuous runs, while the long context ensures task coherence across multi-turn tool calls. Deployment natively supports vLLM, SGLang, and Transformers, and open-source weights meet the private deployment and compliance needs of sensitive industries such as finance. The article also notes that training data composition, long-cycle task evaluation methods, and commercial license terms are all undisclosed, and actual capability still needs community validation on real long tasks.