macro-F1
A multiclass evaluation metric that first computes the F1 score for each class, defined as the harmonic mean of precision and recall, and then averages these scores equally across all classes. Each class therefore receives the same voting weight regardless of its number of samples.
Timeline (last 90 days)
On 300 practice exam papers labeled by Qwen3-32B, fine-tuned v3 of Laya raised the judgment macro-F1 across 11 questions from 0.33 to 0.57 versus the zero-shot version, cut calibration error from 0.15 to 0.07, and reduced danger-level MAE from 2.78 to 1.56.
One of the four categories ("explicitly missing") has no observational support, and its macro-F1 numerical contribution does not equal having measured performance on that state.
When the number of observations for a category is 0, its F1 may formally be defined as 0 or skipped, but there is no basis at all for estimating performance on that category.
Cohen's kappa reaches 0.964 on boundary-matched spans, but drops sharply to a macro F1 of 0.753 after incorporating missed entities, boundary errors, and label corrections.
All Facts (4)
On 300 practice exam papers labeled by Qwen3-32B, fine-tuned v3 of Laya raised the judgment macro-F1 across 11 questions from 0.33 to 0.57 versus the zero-shot version, cut calibration error from 0.15 to 0.07, and reduced danger-level MAE from 2.78 to 1.56.
50%UnverifiedOne of the four categories ("explicitly missing") has no observational support, and its macro-F1 numerical contribution does not equal having measured performance on that state.
50%UnverifiedWhen the number of observations for a category is 0, its F1 may formally be defined as 0 or skipped, but there is no basis at all for estimating performance on that category.
50%UnverifiedCohen's kappa reaches 0.964 on boundary-matched spans, but drops sharply to a macro F1 of 0.753 after incorporating missed entities, boundary errors, and label corrections.
50%