Unverified50% confidenceFactExact time
Most changes to an AI feature (prompt edits, model updates, data changes, cost changes) do not show up in diff review, and the eval set should be run on every such change.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn most cases, when an AI feature is 'almost there,' fine-tuning is the wrong next step; what the team lacks is context, prompts, retrieval, or evaluation, not model weights.73% similarUnverifiedWhen evaluating AI capabilities, the reference value of a single output result is limited, and the way prompts are designed can profoundly influence conclusions.73% similarUnverifiedDependency updates do not change the capabilities or billing of the Voyage AI model; what actually affects output quality is the model parameter specified in the request and the input content.72% similarUnverifiedThe correctness of AI behavior cannot be detected with ordinary unit tests and requires the use of evaluation sets; after a prompt is changed, a vision model may miss half the books while the program does not crash.71% similarUnverifiedIn AI systems, when a metric moves in the wrong direction, the metric itself is usually the bug rather than the model.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1064247API
curl https://kongchang.com/api/v1/knowledge/claims/1064247MCP
get_claim(id=1064247)