Benchmark
When2Call
A benchmark for evaluating the judgment ability of decision models in scenarios of whether to call a tool, ask a question, answer, or refuse.
Timeline (last 90 days)
Oct 13
SubQ Snap scored 67.0 in the When2Call test, losing to a competitor's 74.6.
Unverified50%
Oct 2
In benchmark tests, 2.0 shows notable improvements over the base model on IFBench (37.3→43.7), When2Call (48→58), and BFCL irrelevance (60→78).
Unverified50%