Unverified50% confidenceBenchmarkExact time
After fine-tuning, Laya reaches 0.766 on the same benchmark, versus the third-party-published Jev figure of 0.727; Drex offers fine-tuning as a vendor service.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/19/2026
First Seen
Valid until: 1/17/2027
Sources
Related Entities
Related Claims
UnverifiedBoth Jev and Drex support up to 255 options; Jev reaches 79.2% on the full Banking77 test (77 labels); Laya's performance degrades beyond about 20 options under default settings, reaching only 0.425 on Banking77.70% similarUnverifiedOn calibration, Jev's ECE in independent tests varies between 0.02 and 0.35 depending on the task; Laya's ECE is 0.466 before temperature fitting (overconfident) and drops to 0.081 after fitting; Drex has not published calibration data.64% similarUnverifiedAfter fine-tuning, Laya's accuracy on the MASSIVE dataset can be improved from 0.45 to 0.85, and on another task from 0.362 to 0.766.62% similarUnverifiedCompared with v1.2, v1.5 improves the F0.5 for real spelling errors from 0.882 to 0.963 and the F0.5 for missing words from 0.078 to 0.698; tense errors remain a weak point, with an F0.5 of only 0.119.61% similarUnverifiedLaya achieves only 0.45 accuracy on the MASSIVE dataset covering 13 languages and only 0.425 on Banking77, while JAF reports 0.87 on the same tasks.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1065251API
curl https://kongchang.com/api/v1/knowledge/claims/1065251MCP
get_claim(id=1065251)