ViSkill: Empowering VLM Agents with Vision-Native Skills

ViSkill is a vision-native skill learning framework that lets VLM agents encode successful interactions into composite visual skill cards, invoke them directly during reasoning and reward shaping, and let the skill library and policy reinforce each other through closed-loop feedback. It achieves an overall success rate of 0.89 on Sokoban, FrozenLake, and PrimitiveSkill, rising to 0.91 with cold start.

Why Textualized Skills Lose Information

The core idea behind skill-augmented agents is nothing new: distill successful trajectories into reusable policies to improve sample efficiency. The problem is that most methods remain "text-centric." They linearize spatial layouts, action-state correspondences, and everything else into language descriptions — a process that discards critical geometric structure.

For pushing tasks like Sokoban, the position of a box relative to its target, or the agent's orientation relative to the box, is often far more precise than any verbal description. Compressing this information into text forces the model to mentally reconstruct spatial relationships at inference time, introducing errors and ambiguity.

VLM (Vision-Language Model) is the foundation model underlying frameworks like ViSkill. Unlike traditional text-only LLMs, VLMs are exposed to both images and text during pretraining, giving them the ability to map visual input into language space — and thus to directly "see" spatial relationships in images. Common examples include GPT-4V, Qwen-VL, and LLaVA. The key to ViSkill bypassing the textualization bottleneck lies in storing skill cards in visual form and feeding them to the VLM as image context at inference time, letting the model's own visual encoder handle geometric structure — rather than having an external pipeline translate spatial relationships into text and then asking the model to reconstruct them. This also means the framework's ceiling is constrained to some extent by the spatial perception capability of the chosen VLM.

ViSkill's Approach: Visual Skill Cards and a Closed Loop

ViSkill proposes a vision-native skill learning framework. Instead of writing skills as text, it encodes them as composite visual skill cards that VLM agents can directly access and invoke.

These retrieved skills serve two purposes: guiding reasoning and participating in reward shaping. Meanwhile, successful trajectories are distilled back into the skill library. This creates a closed loop — skill accumulation and policy improvement reinforce each other, rather than developing on separate tracks.

This is precisely the key difference from prior work. Some earlier studies introduced visual evidence, but skill construction and updating were carried out separately from policy optimization, leaving the mutual reinforcement between the two underexplored. ViSkill binds them into the same feedback loop.

Additionally, the framework offers an optional cold-start mechanism to accelerate early-stage learning, allowing the agent to get off to a faster start even when the skill library is not yet rich.

Reward shaping refers to modifying the environment's raw reward signal — without changing the task's ultimate objective — by adding extra guidance terms, so that agents can receive denser learning signals in sparse-reward settings. A classic approach is potential-based reward shaping, which guarantees the optimal policy remains unchanged. What makes ViSkill special is that the extra rewards used for shaping come from retrieved visual skill cards rather than hand-designed heuristic functions. In this way, successful experiences accumulated in the skill library translate directly into policy guidance, placing skill retrieval, reward generation, and policy updates in the same feedback loop to form a self-reinforcing cycle.

Evaluation Results: 0.89 to 0.91

ViSkill was evaluated on three environments: Sokoban, FrozenLake, and PrimitiveSkill.

The overall success rate reaches 0.89; with cold-start initialization enabled, this figure rises to 0.91. The paper claims it outperforms all proprietary and open-source baselines evaluated, while converging faster than standard PPO.

The code is open-sourced on GitHub (ZJU-REAL/ViSkill).

PPO (Proximal Policy Optimization) is one of the most commonly used policy gradient algorithms in reinforcement learning. By constraining the magnitude of each policy update, it balances training stability and sample efficiency, and is widely used as a baseline across various tasks. ViSkill reports faster convergence than standard PPO, meaning it reaches higher success rates in fewer interaction steps. Each of the three evaluation environments has a different focus: Sokoban tests long-horizon planning and spatial reasoning, FrozenLake emphasizes decision-making to avoid traps in stochastic environments, and PrimitiveSkill focuses on the reuse of basic skills. Evaluation across these three task types aims to demonstrate that visual skill cards are not limited to a single task category.

A Few Signals Worth Noting

First, making skills "visual" rather than "textual" is a pragmatic choice for spatial reasoning tasks. Text is not a universal intermediate representation — at least not in scenarios dense with geometric structure.

Second, the closed-loop design makes the skill library not a static knowledge cache but a living entity that evolves alongside the policy. The impact of this on long-term learning efficiency may be more noteworthy than any single success-rate figure.

Third, the +0.02 gain from cold start is modest, but it does show that early guidance has real value. How to design more effective priors during the cold-start phase, when the skill library is sparse, remains an open question.

One-line Summary

ViSkill encodes successful interactions into visual skill cards, letting the skill library and policy reinforce each other through closed-loop feedback.

Paragraph Summary

ViSkill is a vision-native skill learning framework. Addressing the common "text-centric" problem in existing skill-augmented agents — where linearizing spatial relationships into text loses geometric structure — it encodes successful interactions into composite visual skill cards that VLM agents can directly retrieve and invoke. Retrieved skills are used both to guide reasoning and to participate in reward shaping, while successful trajectories are distilled back into the skill library, forming a closed loop in which skill accumulation and policy improvement reinforce each other. This is the key distinction from prior work that separates skill construction from policy optimization. The framework also offers an optional cold-start mechanism to accelerate early learning. On Sokoban, FrozenLake, and PrimitiveSkill, ViSkill achieves an overall success rate of 0.89, rising to 0.91 with cold start, surpassing the proprietary and open-source baselines evaluated, and converging faster than standard PPO. The code is open-sourced on GitHub.