Unverified50% confidenceFactExact time
Heretic 2.0 allows users to freely define scoring criteria by writing scorer plugins, and any number of scorers can be loaded simultaneously with each set to minimize or maximize.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/1/2026
First Seen
Valid until: 1/30/2027
Sources
Related Entities
Related Claims
UnverifiedThis rubric-based reward framework dynamically generates a dedicated rubric for each specific query, i.e., a query-specific rubric generation mechanism66% similarUnverifiedA complete evaluation benchmark usually includes a fixed dataset, ground truth, automated scoring scripts, and a leaderboard, with the goal of producing comparable quantitative scores.64% similarUnverifiedThe hierarchical verifier expands the evaluation result into multiple sub-scores, making the reward signal denser and the source of errors easier to locate, which is more advantageous in reinforcement learning optimization.64% similarUnverifiedBuilt-in evaluators cover general quality dimensions, such as the helpfulness, relevance, and correctness of answers, and are ready to use out of the box.64% similarUnverifiedPractices for improving scoring consistency include: prioritizing binary labels in the early stage, using a separate judging prompt per dimension, writing each criterion as a question answerable solely from the image and the description, including pass and fail examples (especially edge cases), and providing the image to the judge.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1123807API
curl https://kongchang.com/api/v1/knowledge/claims/1123807MCP
get_claim(id=1123807)