DEEPO
A two-stage reinforcement learning optimization method addressing hallucination in multimodal large language models. It operates through two branches triggered by semantic entropy: expert prefix injection and advantage-sign-aware Rényi preprocessing, targeting advantage collapse and gradient saturation during RL training.
Timeline (last 90 days)
There is a paper on arXiv titled "DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs" that proposes a dual-stage enhancement scheme to suppress hallucinations in multimodal large language models.
DEEPO is a dual-stage enhancement method combining signal variance regularization and gradient preprocessing, comprising a semantic-entropy-triggered expert prefix branch and an advantage-sign-aware Renyi preprocessing branch.
DEEPO's expert prefix is a dynamically triggered supervised injection mechanism that injects fact-based guided continuations only when a query's semantic entropy exceeds a threshold, in order to restore the variance of the advantage.
Advantage-sign-aware Renyi preprocessing is used to counteract saturation at the logit level, enabling gradient signals to propagate effectively in high-confidence regions and penalizing confident but incorrect predictions.
On VideoMMMU, the most complex long-horizon task in the evaluation suite, the interaction effect of DEEPO's two branches is statistically significant, with an improvement of +4.0 over the baseline (95% confidence interval [1.1, 6.9]).
On other tasks, the effects of DEEPO's two branches are additive, meaning that combining the two does not cause them to cancel each other out.
DEEPO claims to reduce hallucinations while maintaining model accuracy and training stability.
All Facts (7)
Advantage-sign-aware Renyi preprocessing is used to counteract saturation at the logit level, enabling gradient signals to propagate effectively in high-confidence regions and penalizing confident but incorrect predictions.
50%UnverifiedDEEPO is a dual-stage enhancement method combining signal variance regularization and gradient preprocessing, comprising a semantic-entropy-triggered expert prefix branch and an advantage-sign-aware Renyi preprocessing branch.
50%UnverifiedDEEPO's expert prefix is a dynamically triggered supervised injection mechanism that injects fact-based guided continuations only when a query's semantic entropy exceeds a threshold, in order to restore the variance of the advantage.
50%UnverifiedDEEPO claims to reduce hallucinations while maintaining model accuracy and training stability.
50%UnverifiedOn VideoMMMU, the most complex long-horizon task in the evaluation suite, the interaction effect of DEEPO's two branches is statistically significant, with an improvement of +4.0 over the baseline (95% confidence interval [1.1, 6.9]).
50%UnverifiedOn other tasks, the effects of DEEPO's two branches are additive, meaning that combining the two does not cause them to cancel each other out.
50%UnverifiedThere is a paper on arXiv titled "DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs" that proposes a dual-stage enhancement scheme to suppress hallucinations in multimodal large language models.
50%