[KongchangAI]
· 2 min read· 1,134 words

Large Models Can Still Learn Skills After Data Is Hidden: New Study Reveals General Progress Can Substitute for Specific Training

Large Models Can Still Learn Skills After Data Is Hidden: New Study Reveals General Progress Can Substitute for Specific Training

Models can indirectly acquire hidden skills through general progress on other data; data filtering only delays capability emergence.

A new study hid training signals for specific skills across chess, Othello, molecular strings, bracket languages, and web text to see if models could reacquire them. Results show general progress on remaining data substitutes for hidden data: hidden-skill delays can be predicted from smaller models, an untrained chess model ranks legal moves almost as well as a trained one, and relearning requires only a tiny fraction of data. The finding also holds for a programming language in a 7-billion-parameter model. Data filtering only delays, not prevents, capability emergence.

Language model capabilities derive from training data, and developers often deliberately hide certain data to prevent models from learning skills that could be misused. How long can this delay last? How much data is needed afterward to reacquire the skill? A new study offers a counterintuitive answer: general progress the model makes on the remaining training data can substitute for the hidden data.

Study source screenshot

Experimental Design: Hiding Specific Skill Signals Across Multiple Domains

In models for chess, Othello, molecular strings, bracket languages, and web text, the researchers hid the training signal for a single skill. For example, in the chess model, they removed all training data about responding to "check"; in a 7-billion-parameter model, they hid the training text for a certain programming language.

By comparing model performance before and after hiding, the team was able to quantify the impact of data filtering on capability delay and observe how the model's progress on other tasks affects acquisition of the hidden skill.

Data filtering refers to actively removing or downweighting specific types of data in the training corpus to prevent a model from acquiring certain capabilities. Common approaches include rule-based keyword filtering, classifier-based content screening, and wholesale removal of specific domains (such as code or game records). Data filtering is widely used to reduce the risk of models outputting harmful content or misusing skills, but its effectiveness relies on an assumption: without the corresponding data, the model cannot acquire the capability. This study stress-tests that assumption—by removing only a single skill's signal from the same training distribution, it observes whether the model can "indirectly" recover that capability from the remaining data.

Parameter count refers to the number of trainable weights in a model and is a rough measure of model capacity. In the text, "700 million parameters" and "7 billion parameters" differ by an order of magnitude; this scale difference is used in the study to compare the acquisition speed and delay of the same skill in models of different capacities. Generally speaking, models with more parameters can extract more general representations from the same data, and are therefore more likely to generalize a hidden skill from unrelated data—which is why the researchers set up a small-versus-large model comparison.

Core Finding: General Progress Substitutes for Specific Data

Experiments show that general progress the model gains on the remaining training data can substitute for the hidden data. Specifically:

  • Predictable delay: In larger models, the delay before the hidden skill appears can be predicted from the performance of smaller models.
  • Implicit learning exists: A chess model never explicitly trained to respond to "check" ranks legal moves almost as well as a trained model.
  • Extremely efficient relearning: When models are subsequently exposed to the hidden data, they need only a small fraction of the data required to train from initialization to master the skill, and this fraction continues to decline as the model improves.
  • Difficulty control: A control rule of comparable difficulty is learned much more slowly, indicating that general progress is not a万能 (universal) substitute, but it can indeed specifically substitute for particular data.
  • Programming language validation: A programming language hidden from a 7-billion-parameter model was reacquired in the same way.

These results appear consistently across multiple domains, indicating the phenomenon is general.

Skill signal refers to the sample patterns in training data that directly teach a model a specific capability. For example, move records in chess game data showing "how to respond when in check" and code snippets in a programming language that embody syntax and semantics are both specific skill signals. Removing these signals is not equivalent to removing all related knowledge: chess data still contains many general patterns such as legal moves and position evaluation, and code data still contains syntactic structures of other languages. This is precisely the core mechanism of the study—general regularities the model learns from this "surrounding" data can transfer to the hidden specific skill, forming what is called implicit learning.

Delay here refers to the number of training steps or amount of data required from the start of training until the hidden skill first appears stably. The study finds that the delay can be extrapolated from small models to large models, a property with practical value: before training a large model, small-scale experiments can estimate when a filtered capability will "re-emerge," providing a basis for safety assessment and time-window management.

Implications for Data Filtering Strategies

Data filtering can indeed delay the emergence of a capability, but as the model's overall capabilities improve, less and less data is needed to reacquire that capability. This means:

  1. Filtering is not a permanent barrier: If developers want to prevent a capability over the long term, hiding data alone may not be enough, because the model will "draw inferences" from other data.
  2. Balancing safety and efficiency: Filtering data increases the cost of subsequent fine-tuning, but the cost decreases as model scale increases.
  3. Predicting capability emergence: The performance of small models can be used to predict the delay of hidden skills in large models, offering a new approach for model evaluation.

Limitations and Future Directions

The study focuses mainly on specific skills (such as chess rules and programming languages); whether more complex skills behave the same way remains to be explored. In addition, exactly which data drives the "general progress" in the experiments still requires further analysis. Regardless, this work reveals the dynamic and subtle relationship between data filtering and model capabilities.

For AI developers and safety researchers, this study reminds us that the boundaries of a model's capabilities are not determined by a single data source, but are a product of the overall training distribution. When designing data filtering strategies, model scale and general progress need to be taken into account, rather than simply removing sensitive data.

One-line Summary

A study finds models can indirectly acquire hidden skills through general progress on remaining data; data filtering only delays rather than prevents capability emergence.

Paragraph Summary

A new study hid training signals for specific skills across domains including chess, Othello, molecular strings, bracket languages, and web text, observing whether models could reacquire these skills. Results show that general progress on the remaining data can substitute for the hidden data: the delay before a hidden skill appears can be predicted from smaller models; a chess model never explicitly trained to respond to "check" ranks legal moves almost as well as a trained model; and when re-exposed to the hidden data, only a tiny fraction is needed to master the skill. The phenomenon also holds in a programming language experiment with a 7-billion-parameter model. The study concludes that data filtering only delays capability emergence and cannot permanently prevent it, so developers must factor model scale and general progress into safety strategies.

#data filtering#large language models#implicit learning#model capabilities#AI safety
Share:

Related articles