JHiMA-WebRL
A hierarchical multi-agent reinforcement learning framework for long-horizon web tasks. It decomposes the agent into a task decomposer and a subtask-conditioned executor, and achieves closed-loop collaboration between planning and execution through two-stage training: supervised fine-tuning from a teacher model followed by dual-role GRPO joint optimization.
Timeline (last 90 days)
The JHiMA-WebRL framework comes from the latest research on arXiv and adopts a joint training approach of planner plus executor, enabling 7B-level small models to match or even surpass larger monolithic models on long-horizon web navigation tasks.
JHiMA-WebRL splits the web agent into a task decomposer and a subtask-conditioned actor; the decomposer infers high-level subtasks, and the actor performs visual grounding and concrete operations.
JHiMA-WebRL training has two stages: in the first stage, a strong teacher model is used to construct structured subtask supervision signals and fine-tune the two modules; in the second stage, GRPO is extended to the dual-role setting, jointly optimizing the two modules in the WebArena environment through hierarchical rollout grouping and role-specific advantage estimation.
JHiMA-WebRL extends GRPO to the dual-role setting, requiring reward signals to be defined separately for the decomposer and the actor and role-specific advantage estimation to be performed, while ensuring through hierarchical rollout that samples within the same group come from the same subtask context.
Ablation experiments verified the value of joint updates: updating the decomposer as well during the RL stage performs better than updating only the actor.
The 37.2% success rate shows that long-horizon web tasks are far from solved, and most tasks still fail.
In JHiMA-WebRL, the credit assignment problem refers to how to reasonably attribute the global reward to the two roles of decomposer and actor, and is an important focus for future improvement.
All Facts (7)
The JHiMA-WebRL framework comes from the latest research on arXiv and adopts a joint training approach of planner plus executor, enabling 7B-level small models to match or even surpass larger monolithic models on long-horizon web navigation tasks.
50%UnverifiedJHiMA-WebRL splits the web agent into a task decomposer and a subtask-conditioned actor; the decomposer infers high-level subtasks, and the actor performs visual grounding and concrete operations.
50%UnverifiedJHiMA-WebRL training has two stages: in the first stage, a strong teacher model is used to construct structured subtask supervision signals and fine-tune the two modules; in the second stage, GRPO is extended to the dual-role setting, jointly optimizing the two modules in the WebArena environment through hierarchical rollout grouping and role-specific advantage estimation.
50%UnverifiedJHiMA-WebRL extends GRPO to the dual-role setting, requiring reward signals to be defined separately for the decomposer and the actor and role-specific advantage estimation to be performed, while ensuring through hierarchical rollout that samples within the same group come from the same subtask context.
50%UnverifiedAblation experiments verified the value of joint updates: updating the decomposer as well during the RL stage performs better than updating only the actor.
50%UnverifiedThe 37.2% success rate shows that long-horizon web tasks are far from solved, and most tasks still fail.
50%UnverifiedIn JHiMA-WebRL, the credit assignment problem refers to how to reasonably attribute the global reward to the two roles of decomposer and actor, and is an important focus for future improvement.
50%