Unverified50% confidenceFactExact time
When directly training an investigator model against a safety-aligned target model, the vast majority of attempts receive refusal responses, resulting in an extremely sparse reward signal.
1
Sources
50%
Confidence
Long-term
Relevance
10/30/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedEven models that have undergone the most rigorous alignment training find it difficult to achieve a 100% refusal rate when faced with sufficiently complex adversarial prompts.77% similarUnverifiedRelying solely on model-side safety training is considered insufficient to address the risks posed by agents in real-world environments77% similarUnverifiedThe pretraining corpora of the vast majority of models are closed-source, forcing membership inference research to rely on heuristic methods to construct members and non-members, which may introduce systematic bias.75% similarUnverifiedSecurity teams should establish a refusal-behavior baseline for the models used by agents; if a model's refusal rate for known sensitive categories is abnormally low, a review should be triggered.75% similarUnverifiedThe reinforcement learning modifications tested in the study did not yield consistent gains, and directly training models to learn to abstain proved unstable.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1114192API
curl https://kongchang.com/api/v1/knowledge/claims/1114192MCP
get_claim(id=1114192)