User Context-Aware Evaluation
An evaluation method that treats a user's previously accumulated history, such as age, sex, medical history and medication use, as preconditions for judging whether a response is safe for that specific user. Studies show that, compared with context-blind evaluation, this method reveals more high-risk responses.
Timeline (last 90 days)
When the user's own historical background was fed to the evaluation model (user context-aware evaluation), the high-risk proportion rose by another 13 to 17 percentage points compared with context-blind evaluation.
About a quarter of conversations judged safe in isolated evaluation (context-blind) were actually unsafe for the specific questioner.
Context-blind evaluation treats each question-answer segment as an independent sample, checking only whether the reply itself conforms to medical common sense and safety norms; user context-aware evaluation takes the user's previously accumulated historical information such as age, gender, past medical history, and medication status as preconditions, and then judges whether this reply is safe for this specific person.
All Facts (3)
When the user's own historical background was fed to the evaluation model (user context-aware evaluation), the high-risk proportion rose by another 13 to 17 percentage points compared with context-blind evaluation.
50%UnverifiedAbout a quarter of conversations judged safe in isolated evaluation (context-blind) were actually unsafe for the specific questioner.
50%UnverifiedContext-blind evaluation treats each question-answer segment as an independent sample, checking only whether the reply itself conforms to medical common sense and safety norms; user context-aware evaluation takes the user's previously accumulated historical information such as age, gender, past medical history, and medication status as preconditions, and then judges whether this reply is safe for this specific person.
50%