ZGCM-1
A fully open-source 7-billion-parameter dense foundation model trained from scratch, employing a hybrid architecture of interleaved gated sliding-window attention and full attention, combined with tool calling and internal reasoning, with support for up to 256K context length.
Timeline (last 90 days)
ZGCM-1 is a fully open-source dense base model with 7 billion parameters, trained from scratch.
ZGCM-1-7B remains competitive with large models such as Qwen3-235B-A22B and GLM-5.1 on mathematical reasoning and agentic search tasks.
ZGCM-1 adopts a mixed attention mechanism of interleaved gated sliding-window attention and full attention.
ZGCM-1 supports an ultra-long context of 256K.
ZGCM-1 uses a stable FP8 Muon optimizer, combining the Muon optimizer with FP8 low-precision training while maintaining training stability.
The paper states that ZGCM-1 achieves approximately a 4.2× improvement in time-to-loss efficiency over the baseline during 16K-context pretraining.
ZGCM-1 employs progressive curriculum learning, expanding the context length in stages of 16K, 64K, and 256K.
During the mid-training phase, ZGCM-1 reformulates the agent-tool interaction process as a Markov Decision Process (MDP).
ZGCM-1 establishes an AI-native R&D workflow, using agent clusters to autonomously manage cluster operations, data filtering, and diagnostic evaluation.
The paper distills eight empirical findings covering architectural scaling, SFT data quality pruning, long-context generalization, and agent co-training dynamics.
3 more timeline events
All Facts (13)
ZGCM-1's competitiveness is mainly reflected in specific task types, while general capabilities remain constrained by model scale.
50%UnverifiedZGCM-1-7B remains competitive with large models such as Qwen3-235B-A22B and GLM-5.1 on mathematical reasoning and agentic search tasks.
50%UnverifiedZGCM-1 adopts a mixed attention mechanism of interleaved gated sliding-window attention and full attention.
50%UnverifiedZGCM-1 supports an ultra-long context of 256K.
50%UnverifiedZGCM-1 uses a stable FP8 Muon optimizer, combining the Muon optimizer with FP8 low-precision training while maintaining training stability.
50%UnverifiedThe paper states that ZGCM-1 achieves approximately a 4.2× improvement in time-to-loss efficiency over the baseline during 16K-context pretraining.
50%UnverifiedZGCM-1 employs progressive curriculum learning, expanding the context length in stages of 16K, 64K, and 256K.
50%UnverifiedDuring the mid-training phase, ZGCM-1 reformulates the agent-tool interaction process as a Markov Decision Process (MDP).
50%UnverifiedZGCM-1 establishes an AI-native R&D workflow, using agent clusters to autonomously manage cluster operations, data filtering, and diagnostic evaluation.
50%UnverifiedThe paper distills eight empirical findings covering architectural scaling, SFT data quality pruning, long-context generalization, and agent co-training dynamics.
50%UnverifiedZGCM-1 is a fully open-source dense base model with 7 billion parameters, trained from scratch.
50%UnverifiedThe core proposition of ZGCM-1 is that compact models can break through parameter capacity limitations by coupling deliberate internal thinking with active external tool invocation.
50%UnverifiedZGCM-1 publicly releases model weights, intermediate checkpoints, complete training code, data and data recipes, and W&B training logs for the pretraining, mid-training, and post-training stages.
50%