GSPO
A reinforcement learning policy optimization method that computes importance weights at the whole-sequence level and normalizes them by sequence length, reducing variance in long-sequence training and improving stability. It can be regarded as a sequence-level refinement of GRPO.
Timeline (last 90 days)
PSGAgent adopts a two-stage training pipeline: the first stage builds an error-aware multimodal chain-of-thought (M-CoT) dataset for cold-start learning; the second stage uses a hierarchical reward mechanism combined with group sequence policy optimization (GSPO) to learn the tool-use policy.
Group sequence policy optimization (GSPO) can be seen as an improvement of GRPO in sequence-level importance sampling, computing importance weights at the whole-sequence level and normalizing by sequence length to reduce variance and improve training stability for long-sequence generation.
The team restored accuracy loss to within 1% in stages through reinforcement learning (GSPO), On-Policy Distillation, and a ThinkingCap 3.6 27B adapter, treating On-Policy Distillation as the key step.
All Facts (3)
PSGAgent adopts a two-stage training pipeline: the first stage builds an error-aware multimodal chain-of-thought (M-CoT) dataset for cold-start learning; the second stage uses a hierarchical reward mechanism combined with group sequence policy optimization (GSPO) to learn the tool-use policy.
50%UnverifiedGroup sequence policy optimization (GSPO) can be seen as an improvement of GRPO in sequence-level importance sampling, computing importance weights at the whole-sequence level and normalizing by sequence length to reduce variance and improve training stability for long-sequence generation.
50%UnverifiedThe team restored accuracy loss to within 1% in stages through reinforcement learning (GSPO), On-Policy Distillation, and a ThinkingCap 3.6 27B adapter, treating On-Policy Distillation as the key step.
50%