Unverified50% confidenceOpinionExact time
Prompt engineering at the product level can materially undermine the safety protections established at the training level without modifying model weights.
1
Sources
50%
Confidence
Long-term
Relevance
10/4/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedRelying solely on model-side safety training is considered insufficient to address the risks posed by agents in real-world environments80% similarVerifiedModel-level safety alignment is fragile, because open-source model weights are accessible and behaviors are modifiable, meaning the alignment efforts invested by vendors can be erased by simple downstream operations.77% similarUnverifiedDefending against prompt injection should rely on structural constraints rather than expecting the model to identify attacks on its own, as semantic-layer detection has been proven to be extremely unreliable.74% similarUnverifiedSecurity isolation for sub-agents should adopt physical isolation (directly removing tools) rather than rule-based isolation (documentation constraints), because large language models carry prompt injection risks that may allow rules to be bypassed.73% similarUnverifiedThe fact that models can still cheat on simple variants of evaluations suggests that existing alignment training may carry the risk of overfitting to the form of evaluations, and that models' understanding of objectives is superficial and fragile.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/973195API
curl https://kongchang.com/api/v1/knowledge/claims/973195MCP
get_claim(id=973195)