SimRec
A reinforcement-learning-based training framework for recommendation agents. It treats the complete interaction policy of a tool-integrated recommendation agent, including search, follow-up questioning, and re-recommendation, as the optimization target, employs a task-anchored user simulator and task-aligned rewards, and uses GRPO to optimize the entire trajectory.
Timeline (last 90 days)
SimRec is a reinforcement learning framework whose goal is to train the complete interaction policy of a tool-integrated recommendation agent, rather than separately tuning the search, ranking, and follow-up questioning modules.
SimRec introduces a task-anchored user simulator that does not rely on user profiles but instead conditions its feedback on the current task requirements.
SimRec's task-aligned reward uses GRPO to optimize the entire trajectory, with rewards including the final hit result, natural retrieval quality, and effective tool use.
SimRec employs a training-only goal injection mechanism to alleviate the sparse reward problem.
Goal injection works by providing the agent or simulator with additional partial information about the target item during training (such as category and attribute keywords), artificially increasing the reachability of the task, and it is used only during training and completely removed at inference.
On the ESCI dataset, SimRec raises a Qwen3-4B agent's NDCG@100 from 32.23% to 35.42%.
The trained 4B model surpasses much larger training-free agents in NDCG and also outperforms task-aligned baseline methods.
The trained agent is noticeably more frequent in three behaviors: verifying candidate items, leveraging user preferences, and recovering from rejected recommendations to offer a new plan.
The SimRec code is available at https://github.com/ZJU-REAL/SimRec.
The gap between the simulator and real users, as well as the feedback quality in self-play mode, remain aspects of this type of method that require continuous validation in deployment.
All Facts (10)
The gap between the simulator and real users, as well as the feedback quality in self-play mode, remain aspects of this type of method that require continuous validation in deployment.
50%UnverifiedSimRec introduces a task-anchored user simulator that does not rely on user profiles but instead conditions its feedback on the current task requirements.
50%UnverifiedSimRec's task-aligned reward uses GRPO to optimize the entire trajectory, with rewards including the final hit result, natural retrieval quality, and effective tool use.
50%UnverifiedSimRec employs a training-only goal injection mechanism to alleviate the sparse reward problem.
50%UnverifiedGoal injection works by providing the agent or simulator with additional partial information about the target item during training (such as category and attribute keywords), artificially increasing the reachability of the task, and it is used only during training and completely removed at inference.
50%UnverifiedOn the ESCI dataset, SimRec raises a Qwen3-4B agent's NDCG@100 from 32.23% to 35.42%.
50%UnverifiedThe trained 4B model surpasses much larger training-free agents in NDCG and also outperforms task-aligned baseline methods.
50%UnverifiedThe trained agent is noticeably more frequent in three behaviors: verifying candidate items, leveraging user preferences, and recovering from rejected recommendations to offer a new plan.
50%UnverifiedSimRec is a reinforcement learning framework whose goal is to train the complete interaction policy of a tool-integrated recommendation agent, rather than separately tuning the search, ranking, and follow-up questioning modules.
50%UnverifiedThe SimRec code is available at https://github.com/ZJU-REAL/SimRec.
50%