[KongchangAI]
ConceptCausality Reward on Scrambled CoTs

CRoS

A metric for evaluating the causality of chain-of-thought (CoT). It perturbs the CoT by shuffling the order of reasoning steps without changing their content, avoiding the confounding effect of information changes, and can serve as an auxiliary reward signal in reinforcement learning.

Timeline (last 90 days)

Nov 2

The researchers propose CRoS (Causality Reward on Scrambled CoTs), a causality metric that perturbs the CoT by scrambling the order of reasoning steps without changing their content.

Unverified50%
Nov 2

On the Reasoning Gym benchmark, using the Gemma-3-4B model, CRoS achieves 46.8% accuracy on five unseen logic and game datasets, surpassing the strongest baseline (43.8%).

Unverified50%
Nov 2

In terms of CoT quality, CRoS consistently outperforms the baseline on CoT-Pass@K (K=1 to 128), averaging 43.4% versus 41.4%.

Unverified50%
Nov 2

For CoT faithfulness, CRoS achieves a prompt-acknowledgment score of 34.0%, compared with 20.2% for the baseline.

Unverified50%
Nov 2

CRoS's order-scrambling perturbation may not apply to all types of reasoning tasks, for example those in which the order of steps itself does not matter.

Unverified50%

All Facts (5)

Source Articles