[KongchangAI]
ConceptHuman Annotation / 人工标注

Human Annotation

The stage in a machine learning pipeline in which trained personnel read, classify, score or correct data samples to produce high-quality labels for training and evaluating models or for diagnosing defects. In the era of large models, it extends to human scoring of model outputs for usefulness, safety and factuality.

Timeline (last 90 days)

Nov 9

Human annotation is a common step in machine learning pipelines, in which trained personnel read, classify, score or correct data samples, producing high-quality labels used to train models, evaluate model outputs, or investigate model defects.

Unverified50%
Nov 9

In the era of large models, human annotation also includes manual scoring of the helpfulness, safety and factuality of model responses (often referred to as human preference data in RLHF).

Unverified50%
Oct 22

Existing methods for obtaining process labels rely on manual annotation, rollout-based estimation, or auxiliary verification, which are costly and difficult to scale.

Unverified50%
Oct 19

The research team first manually annotated a portion of the parliamentary records as ground truth, then had the LLM perform event extraction on the same batch of texts, and calibrated the model outputs by computing metrics such as precision and recall to verify extraction quality.

Unverified50%
Oct 17

When reviewing samples that mix real and synthetic text, human annotators anchor on the surface fluency of synthetic text, thereby systematically favoring synthetic samples.

Unverified50%
Oct 15

The project combines four types of evaluation methods: rule-based deterministic checks, LLM-based judge evaluation, sampling evaluation based on manual annotation, and online evaluation based on user feedback.

Unverified50%

All Facts (6)

Source Articles