Baseten Releases joey-1: A 2B Decision Model That Outputs Probabilities at $0.01 per Million Tokens

Why a Model That Doesn't Write Text Is Worth Noticing

Baseten has released joey-1, an open-weights model with 2B parameters. What sets it apart is that its outputs consist of calibrated probabilities rather than text. According to MarkTechPost, the model has a P50 latency of 20 milliseconds and an inference cost of $0.01 per million input tokens.

Taken together, these three numbers point in the same direction: turning the "judgment" of large models into cheap, fast infrastructure that can plug directly into business systems—not just a conversational interface.

What It Means to Output Probabilities Instead of Text

The fundamental behavior of mainstream language models is token continuation. Even if you ask a yes/no question, it generates text and leaves you to parse it yourself. joey-1 skips that step and directly provides calibrated probability values.

For decision systems, this difference matters more than parameter count. In scenarios like risk control, ranking, content grading, and routing, what's really needed is "how likely is this to belong to a certain category"—and whether that probability is trustworthy. Calibration means the model's output of 0.8 should roughly correspond to an 80% positive rate, so that business teams can set thresholds and calculate expected loss accordingly.

According to MarkTechPost, joey-1 is released as open weights, leaving room for teams that need private deployment or deep customization of model behavior.

Probability calibration is an evaluation dimension of classification models that is independent of accuracy. A model can rank positives and negatives very well (good discrimination), yet its output of 0.8 may not actually correspond to an 80% positive rate—this depends on the training objective and post-processing. Common approaches include fitting Platt scaling (logistic regression calibration) or isotonic regression on a held-out set, and temperature scaling to address overconfidence in neural networks. Whether calibration holds is typically measured with reliability diagrams, expected calibration error (ECE), or Brier score. For business teams, only well-calibrated probabilities can be directly plugged into expected-loss formulas for threshold decisions; uncalibrated scores are suitable only for ranking, not for pricing or credit limit calculations.

Open weights means the model parameters can be downloaded, self-deployed, and fine-tuned, but the training data, training code, or full training pipeline may not be public—so it is not strictly equivalent to "open source model." For teams that need data to stay on-premises or want to recalibrate and fine-tune on their own distribution, open weights are a prerequisite; but it also means no vendor SLA or ongoing maintenance, with deployment, inference optimization, and version management falling on you.

The Logic Behind 2B Parameters and 20ms Latency

2B is a scale reached after a clear trade-off. It cannot compete with general-purpose large models on breadth of knowledge or depth of reasoning, but it gains orders of magnitude in latency and cost.

A P50 latency of 20 milliseconds means it can be embedded in online request paths rather than existing as an asynchronous task. The pricing of $0.01 per million input tokens makes "calling the model on every request" economically viable. For many teams, the main concern about using models in decision-making has been latency and cost, not effectiveness itself.

The sensible positioning of this combination is as a judgment layer within a larger system: an upstream large model handles understanding and generation, while joey-1 handles low-cost, high-frequency classification and scoring.

P50 latency refers to the 50th percentile (median) end-to-end response time—half of requests are faster than this value. It is suitable for describing typical experience but masks the long tail: what online decision paths are really sensitive to is P95 or P99, because timeouts and tail latency often determine whether the system needs degradation or fallback rules. A 20ms P50 with a heavy tail still needs timeout truncation and default policies. Moreover, latency is strongly correlated with throughput, batch size, hardware, and sequence length, so numbers across environments should not be compared directly.

Cost per million tokens is the current standard metric for inference economics, making it easy to compare small and large models. But note that it only covers input-side pricing; output, cache hits, cold starts, and operational costs need separate accounting. For high-frequency short-input scenarios, the actual dominant cost per call may be fixed overhead rather than token count.

Applicable Scenarios and Boundaries

The advantageous scenarios for this type of model are fairly clear: online, low-latency probability judgments whose results directly drive business actions. What these tasks share is relatively structured input, a limited output space, and a requirement for interpretable confidence.

The boundaries are equally clear. It is not suited for open-ended generation, multi-step reasoning, or tasks requiring broad world knowledge. Treating it as a replacement for general-purpose models is a classic misuse.

Additionally, the original article does not disclose the training data composition, calibration method, or evaluation benchmarks, so whether its probability calibration is stable on real business distributions still needs to be verified by users with their own data. This is precisely where the benefit of open weights lies: the cost of verification is controllable.

Summary

The value of joey-1 lies in separating "decision-making" from large models and making it a standalone component that is cheap, fast, and probability-readable. For teams looking for a judgment layer for their online systems, this type of model offers a new option—provided they understand what it is good at and what it is not.

One-line Summary

Baseten releases joey-1, a 2B open-weights model that directly outputs calibrated probabilities, positioned as a low-latency, low-cost judgment layer.

Paragraph Summary

Baseten has released joey-1, a 2B open-weights model whose core difference is that it does not generate text but directly outputs calibrated probability values, with a P50 latency of 20 milliseconds and a cost of $0.01 per million input tokens. The article argues that for decision scenarios such as risk control, ranking, content grading, and routing, "what is the probability of a category and is that probability trustworthy" matters more than parameter count; calibration enables business teams to set thresholds and calculate expected loss. The 2B scale is a clear trade-off: giving up breadth of knowledge and depth of reasoning in exchange for latency and cost advantages that allow embedding in online request paths, positioned as a judgment layer in a larger system where an upstream large model handles understanding and generation. The boundaries are equally clear: not suited for open-ended generation, multi-step reasoning, or broad world-knowledge tasks; and since the original article does not disclose training data, calibration method, or evaluation benchmarks, actual calibration stability needs to be verified by users with their own data—open weights make this verification cost manageable.