EWM

The Explanatory World Model (EWM) is an attribution framework whose core idea is to use explanations to identify early events that may have contributed to the final outcome.

The research team instantiated EWM with frontier language models, using the explanations generated by the language model directly as learning signals.

EWM's conservative hierarchical operator recursively redistributes credit from the outcome timestep back to relevant early actions, while keeping the undiscounted cumulative return for the entire episode unchanged.

EWM operationalizes three complementary explanation variants: actual causality, parsimony of explanation, and hard-to-vary explanations.

On two long-horizon RL benchmarks, Crafter and WordCraft, EWM significantly outperformed two baseline methods, PPO and LaRe.

EWM achieved the greatest performance gains when reward delay was most severe.

On the public sequential recommendation benchmark KuaiRand, EWM surpassed SFT and LaRe in the offline setting and outperformed the A2C algorithm in the online setting.

In offline evaluations of an industrial-grade generative recommendation system, EWM substantially improved recall and enhanced multiple behavioral proxy metrics.

LaRe is the most directly comparable baseline method to EWM, also leveraging language models to assist with reward shaping, but adopting a direct reward relabeling strategy rather than a conservative hierarchical operator.