Unverified50% confidenceFactExact time
A contemporaneous study by O'Brien et al. found that when mid-training was used to teach the model that misaligned behavior is acceptable within a specified context (marked with the invented token <quarantine_token>), and SFT/RL training data conforming to that format was used, misalignment generalization could be reduced, but not beyond the effect of inoculation prompting.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe core idea of inoculation prompting is: if the model is told before training that the behavior is acceptable, then the association between the behavior and misalignment will not be established, thereby blocking the generalization of misalignment; unlike SDF, inoculation prompting is given directly in the form of a system prompt during RL training and involves no parameter updates, so its coverage strength is higher.80% similarUnverifiedDuring fine-tuning, the prompt format in the training phase must strictly match the inference phase; otherwise, performance degradation will occur without any error being reported.76% similarUnverifiedThe overall training process first calibrates the relative relationship between label scores and feature scores, then increases the attention scale at a rate of t^(1/4), causing the overall threshold error to converge at a rate of O(t^(-1/4))76% similarUnverifiedFor classification tasks with discrete and closed label spaces, the training signal from SFT is already clear enough, and introducing reinforcement learning would instead increase the difficulty of hyperparameter tuning and training instability76% similarVerifiedDuring the post-training phase, an excessively low loss may correspond to overfitting, where the model memorizes the training data and loses generalization ability.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012021API
curl https://kongchang.com/api/v1/knowledge/claims/1012021MCP
get_claim(id=1012021)