Factual Consistency
A metric measuring whether model output is consistent with retrieved knowledge. It is used to distinguish whether the root cause of a problem is insufficient capability of the base model or a need to optimize the recall and ranking strategy.
Timeline (last 90 days)
Factual consistency evaluates whether a summary preserves the meaning and context of the original information; scientific conclusions often carry conditional qualifiers, and stripping those qualifiers in a summary may cause the conclusions to be misstated.
Faithfulness corresponds to the concept of factual consistency (Faithfulness / Factual Consistency) in natural language generation evaluation, and differs subtly from truthfulness: truthfulness means whether the answer matches facts in the objective world, while faithfulness only requires that the answer does not go beyond the given context; therefore faithfulness is a more measurable and more controllable metric, whereas truthfulness also depends on the reliability of the retrieval source itself.
All Facts (2)
Factual consistency evaluates whether a summary preserves the meaning and context of the original information; scientific conclusions often carry conditional qualifiers, and stripping those qualifiers in a summary may cause the conclusions to be misstated.
50%UnverifiedFaithfulness corresponds to the concept of factual consistency (Faithfulness / Factual Consistency) in natural language generation evaluation, and differs subtly from truthfulness: truthfulness means whether the answer matches facts in the objective world, while faithfulness only requires that the answer does not go beyond the given context; therefore faithfulness is a more measurable and more controllable metric, whereas truthfulness also depends on the reliability of the retrieval source itself.
50%