Laya-multilingual
A 322M-parameter multilingual text classification model built on mmBERT, used to assess subtext and danger levels in conversations. It outputs probabilities rather than generating text.
Timeline (last 90 days)
Laya-multilingual is a 322M-parameter, Apache-2.0-licensed model built on mmBERT that outputs probability judgments for chat-card questions.
On 300 practice exam papers labeled by Qwen3-32B, fine-tuned v3 of Laya raised the judgment macro-F1 across 11 questions from 0.33 to 0.57 versus the zero-shot version, cut calibration error from 0.15 to 0.07, and reduced danger-level MAE from 2.78 to 1.56.
Fine-tuned v3 achieved a safety recall of 13/21 (62%) at the 0.8 alert threshold, versus 0/21 for zero-shot; false alarms on ordinary chat dropped from 2.9% to 1.4%.
The author acknowledges that fine-tuned v3 failed two preset thresholds: safety recall of 62% is below the 90% target, and F1 on the "whether the crisis has passed" question fell by 0.08, exceeding the 0.02 limit.
The fine-tuning experience shows that F1 varies by 0.02-0.05 across random seeds for the same version, so the average of three seeds should be compared rather than the best seed.
Raising the share of risky-conversation samples from 10% to 25% improves standalone safety-check recall (53% to 68%) but causes the model to raise more false alarms on ordinary chat; using risky samples only for training the safety question can mitigate the side effects.
All Facts (6)
Laya-multilingual is a 322M-parameter, Apache-2.0-licensed model built on mmBERT that outputs probability judgments for chat-card questions.
50%UnverifiedOn 300 practice exam papers labeled by Qwen3-32B, fine-tuned v3 of Laya raised the judgment macro-F1 across 11 questions from 0.33 to 0.57 versus the zero-shot version, cut calibration error from 0.15 to 0.07, and reduced danger-level MAE from 2.78 to 1.56.
50%UnverifiedFine-tuned v3 achieved a safety recall of 13/21 (62%) at the 0.8 alert threshold, versus 0/21 for zero-shot; false alarms on ordinary chat dropped from 2.9% to 1.4%.
50%UnverifiedThe author acknowledges that fine-tuned v3 failed two preset thresholds: safety recall of 62% is below the 90% target, and F1 on the "whether the crisis has passed" question fell by 0.08, exceeding the 0.02 limit.
50%UnverifiedThe fine-tuning experience shows that F1 varies by 0.02-0.05 across random seeds for the same version, so the average of three seeds should be compared rather than the best seed.
50%UnverifiedRaising the share of risky-conversation samples from 10% to 25% improves standalone safety-check recall (53% to 68%) but causes the model to raise more false alarms on ordinary chat; using risky samples only for training the safety question can mitigate the side effects.
50%