Unverified50% confidenceFactExact time
Custom evaluations are check items constructed in reverse from production failure samples, each corresponding to an error that actually occurred, with explicit inputs, expected behavior, and judgment criteria.
1
Sources
50%
Confidence
Long-term
Relevance
11/4/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedError analysis is a methodology for systematically examining failure cases; the process involves collecting failure samples, categorizing them by error type, estimating frequency and impact, and prioritizing the bottlenecks with the highest proportion.78% similarUnverifiedArtificially planted errors are a common evaluation method for testing whether AI has the ability to detect problems.76% similarUnverifiedThe NVIDIA approach recommends converting each category of failure sample into an evaluation item to build a custom evaluation set, so that after modifying prompts, tool-calling strategies, models, or memory mechanisms it can be rerun to judge whether the fix is effective and whether it introduces regressions.76% similarUnverifiedValidation must occur before the workflow executes critical external operations, and records that fail validation should be routed to an error handling path or a manual review queue.75% similarUnverifiedThe reflection mechanism is activated when static checks find problems or dynamic verification such as unit tests fails, reading the failure logs, error messages, and evaluation reports, analyzing the root cause and repair plan, and guiding the executor's next round of modifications.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1136131API
curl https://kongchang.com/api/v1/knowledge/claims/1136131MCP
get_claim(id=1136131)