Bespoke Labs
An AI company that develops the Nimble series of decision models, focused on decision-layer technologies such as model routing and task orchestration. It serves developers and teams building AI systems.
Timeline (last 90 days)
Currently all performance data for Nimble-v4 comes from official disclosures by Bespoke Labs, and there has been no independent third-party reproduction.
According to officially published data, Nimble-v4-max scores 10.5 points higher than Jev 1.13 on the public Decision Index score.
Bespoke Labs' internal evaluation shows that Nimble reduces the error rate by 43% compared with Jev in actual decision-making scenarios.
Bespoke Labs' internal evaluation shows that Nimble improves accuracy by 50 percentage points compared with Jev on the model routing decision task.
Bespoke Labs released the latest generation of models in the Nimble series, Nimble-v4 and Nimble-v4-max, on Product Hunt.
On Bespoke Labs' leakage-level evaluation, Nimble reaches 90.12, while its base model Qwen 3.5 9B scores only 66.36.
The Jeff model is a decision-specific model series developed in-house by Bespoke Labs, with parameter counts ranging from 0.8B to 1.4B, while the Nimble series is fine-tuned on a Qwen base.
The benchmark published by Bespoke Labs covers 33 public datasets and 3,880 manually labeled decisions; Jeff 1.1B achieves the highest average accuracy at 76%, Nimble 9B at 75.7%, Jeff 1.4B at 73.3%, and the 0.8B version at 63.5%.
All Facts (8)
Currently all performance data for Nimble-v4 comes from official disclosures by Bespoke Labs, and there has been no independent third-party reproduction.
50%UnverifiedAccording to officially published data, Nimble-v4-max scores 10.5 points higher than Jev 1.13 on the public Decision Index score.
50%UnverifiedBespoke Labs' internal evaluation shows that Nimble reduces the error rate by 43% compared with Jev in actual decision-making scenarios.
50%UnverifiedBespoke Labs' internal evaluation shows that Nimble improves accuracy by 50 percentage points compared with Jev on the model routing decision task.
50%UnverifiedThe benchmark published by Bespoke Labs covers 33 public datasets and 3,880 manually labeled decisions; Jeff 1.1B achieves the highest average accuracy at 76%, Nimble 9B at 75.7%, Jeff 1.4B at 73.3%, and the 0.8B version at 63.5%.
50%UnverifiedOn Bespoke Labs' leakage-level evaluation, Nimble reaches 90.12, while its base model Qwen 3.5 9B scores only 66.36.
50%UnverifiedThe Jeff model is a decision-specific model series developed in-house by Bespoke Labs, with parameter counts ranging from 0.8B to 1.4B, while the Nimble series is fine-tuned on a Qwen base.
50%UnverifiedBespoke Labs released the latest generation of models in the Nimble series, Nimble-v4 and Nimble-v4-max, on Product Hunt.
50%