AdvRole

AdvRole is a framework based on adversarial context rewriting that transforms reinforcement learning for role-playing from static training into a closed-loop curriculum, conducting adversarial gameplay through the alternating operation of two roles: Actor and Rewriter.

In AdvRole, the Actor is responsible for learning role-playing tasks, while the Rewriter specializes in editing character profiles and dialogue contexts, rewriting them into high-difficulty scenarios targeting the current Actor.

AdvRole's Rewriter is trained with a performance-gap reward: if a rewrite causes the current Actor's score to drop significantly compared to the original scenario, that rewrite receives a higher reward.

The research team evaluated AdvRole on three role-playing benchmarks, covering both English and Chinese language scenarios.

The research team released a new multilingual benchmark to supplement existing evaluations' shortcomings in linguistic diversity.

Experimental results show that AdvRole consistently outperforms baseline methods across all benchmarks.

AdvRole's Actor-Rewriter structure more closely resembles a self-play adversarial curriculum than a sample generation framework, with the goal of having the Rewriter generate scenarios that cause the Actor to lose points.

As a preprint, AdvRole still awaits broader reproduction and scrutiny; stability issues common to adversarial training and the risk of the Rewriter generating unreasonable scenarios are directions that require further observation.