Unverified50% confidenceFactExact time
The study used a label-blind LLM judge to classify failed attacks, determining whether the agent "ignored" or "refused" the injected instruction.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe drawback of the LLM-as-a-Judge approach is that the evaluation itself consumes computing resources, and the judging model may introduce its own biases.76% similarUnverifiedFlaws in the evaluator interface can masquerade as reasoning failures, and failing to distinguish between the evaluation pipeline and model capability when interpreting benchmark results may lead to misleading conclusions.68% similarUnverifiedA controlled "source migration" experiment showed that judges tend to accept erroneous values visible in the input while rejecting correct values whose source is hidden, i.e., judges are "following the reference" rather than independently verifying the correctness of values.68% similarUnverifiedThe strategy of directly rejecting at the history level while issuing warnings and ignoring at the request level balances runtime robustness with data reliability and is a deliberate trade-off design.68% similarUnverifiedUncensored fine-tuning typically involves secondary fine-tuning of a base model on datasets that contain no refusal examples, or penalizing unnecessary refusal behavior during the DPO stage.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1065979API
curl https://kongchang.com/api/v1/knowledge/claims/1065979MCP
get_claim(id=1065979)