SR-Text
A text evaluation set built by the vLLM Semantic Router team for semantic routing scenarios, used to assess model performance on candidate answer selection or routing-target judgment tasks; its scale, construction method and baselines are not publicly disclosed.
Timeline (last 90 days)
Officially, d3-ultra is disclosed to have achieved a score of 87.2 on the team's self-built SR-Text test.
The size of the SR-Text test set, how it was constructed, the number of answer candidates, and the specific metric used (such as accuracy or AUC) are all undisclosed.
There are often differences in criteria between self-built evaluation sets and public benchmarks, and factors such as the difficulty distribution of the test set and the number of answer candidates can significantly affect absolute scores, so 87.2 is more suitable as an internal acceptance signal for the team rather than a basis for cross-comparison.
All Facts (3)
Officially, d3-ultra is disclosed to have achieved a score of 87.2 on the team's self-built SR-Text test.
50%UnverifiedThe size of the SR-Text test set, how it was constructed, the number of answer candidates, and the specific metric used (such as accuracy or AUC) are all undisclosed.
50%UnverifiedThere are often differences in criteria between self-built evaluation sets and public benchmarks, and factors such as the difficulty distribution of the test set and the number of answer candidates can significantly affect absolute scores, so 87.2 is more suitable as an internal acceptance signal for the team rather than a basis for cross-comparison.
50%