Can the Skill Documents Hidden in LoRA Weights Be Read Back Out?

Title

Can the Skill Documents Hidden in LoRA Weights Be Read Back Out?

Meta Description

Research explores whether skill documents can be recovered from LoRA adapters: reading weights directly is nearly useless, but an executing writer can recover 39-44% of key instructions; behavioral adapters require reconstructing skills by comparing outputs between the base and adapted models, enabling cross-model transfer.

Article Content

Compiling skill documents into LoRA saves precious context window space, but the cost is obvious: adapters become hard to read, edit, and audit, and cannot be transferred to other models. A natural question follows—can the original skill document be written back out from the adapter and its frozen base model?

This research offers a layered answer: reading weights alone yields almost nothing, but letting the model run first and then write works far better.

Why Compress Skill Documents into LoRA

Agent skills are usually short documents that teach a language model how to handle a particular class of tasks. To save context, more and more work compiles these documents directly into low-rank adapters—behavior is preserved, but the text disappears.

This creates a set of practical problems. An adapter is a weight file that humans cannot read; changing a single instruction requires retraining. Auditing what it learned has no readable entry point. And using the same capability on a different base model requires re-adaptation.

The research question is: given an adapter and a frozen base model, can the skill document be written back out? Evaluation has two dimensions—how many key instructions from the original document the recovered document covers, and to what extent it solves the original task.

LoRA (low-rank adapter) is a parameter-efficient fine-tuning method: all base model weights are frozen, and two small matrices (a low-rank decomposition) are injected alongside some layers, with only these two matrices updated during training. Since the added parameters are typically only one-thousandth to one-hundredth of the original model, LoRA files are small, fast to load, and pluggable, making them well suited to carrying narrow-domain behaviors like "skill documents." But precisely because knowledge is compressed into a low-rank subspace, the numerical weights of a LoRA do not retain natural-language form—humans cannot read it like a prompt, nor convert it directly into another model's input. Understanding this is key to seeing why "writing the document back out" is a meaningful inverse problem rather than a simple format conversion.

Reading Weights Directly Doesn't Work—Running Does

Researchers first tried methods that map weights directly to text, with poor results: reading weights alone recovers very little. Even a learned weight-to-text mapping transmitted only about one bit of document information.

Switching to letting the adapter actually execute the task changes things markedly.

The Executing Writer they trained is a small writer model built on top of the adapted model. On held-out skills, it recovers 39-44% of key instructions; by contrast, an adapter for a different skill recovers only 1.3-1.5%. On adapters adapted to 41 external skills, the recovered document restores 85-91% of the original task gain.

These strongest results have a precondition: the adapter actually stored document content during training. For adapters trained only on behavior, the trained writer struggles much more, recovering only 1-2% of instructions.

"One bit of document information" refers to the recoverable amount in the information-theoretic sense: if a skill document contains dozens of key instructions, ideal recovery should transmit hundreds of bits; a weight-to-text mapping transmitting only about 1 bit means the model can barely distinguish "has document" from "no document," with output close to random guessing. This is not because the mapping model is too small, but because low-rank adapters compress the document into a behavioral abstraction during training—there is no verbatim encoding in weight space. In contrast, letting the adapted model actually execute tasks is like re-"probing" the adapter's internal behavioral function with input-output pairs, so the information content is naturally much larger.

Behavioral Adapters: Write by Comparing the Differences Between Two Models

For behavioral adapters, the research tested an agent approach: run the base model and the adapted model simultaneously, and infer what the skill should say from the differences in their outputs.

On the ALFWorld environment, skills written this way recover 58-71% of a given LoRA's gain, and can transfer to models that cannot load that LoRA—lifting Llama-3.1-8B from 5% to 55%.

This comparison is worth noting: for the same skill, the transfer value sometimes comes precisely from the fact that the adapter itself cannot be loaded directly.

ALFWorld is a text-based embodied intelligence benchmark: agents complete multi-step tasks such as finding objects, heating, and cleaning in a simulated home environment based on natural-language instructions. It is often used to measure whether "skill documents" truly improve task success rates, because tasks require combining multiple sub-instructions and the base model alone easily gets lost. This paper uses ALFWorld for transfer experiments precisely because it can turn "document quality" into a quantifiable success-rate difference: the same reconstructed skill, loaded onto a model that cannot use the original LoRA, still delivers a 5%→55% improvement, showing that text-form skills are more cross-model portable than weight-form ones.

Who Should Write the Skill Depends on What the Adapter Learned

A more fundamental conclusion is that whether a skill truly "comes from the adapter" depends on what the adapter learned.

When the adapter's behavior is hard to guess, the adapter itself is the information source, and equivalent skills cannot be written without it; when the behavior can be guessed, a reward loop that does not connect to the adapter can achieve similar results.

In other words, the value of an adapter lies not only in its weights but also in the readable document it carries. Compiling the document into weights saves context, but the most effective path to recovering auditable, transferable capability is to let the model run first, then reconstruct the text from behavioral differences.

For agent engineers, this offers a pragmatic direction: rather than trying to "decode" skills from weights, treat the adapter as a source of behavioral samples and use a dedicated small writer or difference-comparison agent to reconstruct readable skill documents.

One-line Summary

Research reveals: recovering skill documents from LoRA—reading weights directly is nearly useless, but letting the model execute tasks and then reconstructing works markedly better.

Paragraph Summary

This article explores how to recover skill documents compiled into low-rank adapters (LoRA) and their frozen base models. The research finds that reading weights directly transmits only about one bit of information and can barely recover text. In contrast, training a small "executing writer" to let the adapted model run and then write recovers 39-44% of key instructions on adapters that stored documents, restoring 85-91% of task gain; adapters trained only on behavior require reconstructing by comparing output differences between base and adapted models, recovering 58-71% of gain on ALFWorld, and the reconstructed skills can transfer to models that cannot load the original LoRA. The conclusion: whether a skill can be recovered depends on what the adapter learned, and the pragmatic path is to let the model run first, then reconstruct readable documents from behavioral differences.