THX-01: A 322M-Parameter Open-Source Decision Model That Completes Inference in 10 Milliseconds

A small model with just 322 million parameters is challenging conventional assumptions about how AI inference should work. HAL-X AI has open-sourced a non-autoregressive decision model called THX-01, and released a notable milestone: since launch, its free API has processed over 70 million tokens. The message is clear — not every AI task requires a multi-billion-parameter large language model generating answers token by token.

Why We Need a Dedicated Decision Model

Most AI applications don't actually require long-form generation. Developers often just need reliable judgment: Which department should this ticket go to? Is this message spam? What number appears in this document? Which passage supports this answer? And — how confident is the model in its own judgment?

Conventional large models generate answers token by token, which is slow and wasteful for structured tasks like classification and decision-making. THX-01 takes a fundamentally different approach: it takes a document or message along with a typed question as input, and returns a structured answer with calibrated probabilities in a single forward pass.

According to official GPU benchmarks, a single inference takes approximately 10 milliseconds — a compelling number for production environments requiring high concurrency and low latency.

The non-autoregressive architecture is the key to understanding THX-01's speed. Traditional autoregressive language models must generate tokens serially — each token depends on all previously generated tokens, making latency proportional to output length. Non-autoregressive models break this serial dependency, enabling parallel output of complete results in a single forward pass. The trade-off is that they typically struggle with generating long, coherent text — but for decision tasks that only need to return a class label, a probability value, or a short excerpt, this trade-off is entirely reasonable. THX-01 precisely matches its architectural choice to its task type: it sacrifices general-purpose generation capability in exchange for extreme speed on structured decision tasks.

Benchmark Results vs. TypeSafe Jev

HAL-X AI compared THX-01 against TypeSafe Jev 1.13 on an internal benchmark of 2,843 multilingual customer service tickets:

Metric	THX-01	Jev 1.13
Accuracy	98.4%	97.4%
Calibration Error (ECE)	0.003	0.007
Measured Latency	~10 ms	331 ms
Open-Source Weights	Yes	No

On this classification benchmark, THX-01 outperforms Jev on both accuracy and calibration error, with measured latency approximately 33× lower. However, the team is upfront about a caveat: the latency figures come from different serving environments and are not a strict apples-to-apples comparison on identical hardware, so that multiplier shouldn't be over-interpreted.

Setting aside the rigor of the latency comparison, the core point still stands — small, specialized decision models don't need billions of parameters to deliver competitive performance on targeted tasks. This aligns with the broader industry conversation around the resurgence of small, vertically focused models.

RLCD: Teaching the Model How Confident to Be

The most noteworthy technical detail in THX-01 is its training method: RLCD (Reinforcement Learning for Calibrated Decisions).

Unlike conventional approaches that only optimize for prediction correctness, RLCD also incentivizes the model to output probability estimates that genuinely reflect its own uncertainty. In other words, the model doesn't just give an answer — it gives a trustworthy confidence score.

This matters enormously for AI agents and production applications. When an automated system receives a prediction, it needs to decide: accept it directly, request more information, or escalate to a more capable model? This kind of decision chain only works reliably when confidence scores are well-calibrated. The extremely low calibration error (ECE 0.003) is a direct reflection of this design goal.

Six Supported Decision Types

THX-01 covers a comprehensive range of structured decision scenarios:

• Choice: Classification across dynamically provided options
• Yes/No: Binary judgment with confidence scores
• Score: Ordinal assessment
• Number: Extracting numerical values from documents
• Excerpt: Extracting precise text passages
• Citations: Identifying supporting evidence

The model supports 18 post-training languages, including English, Russian, Turkish, and Azerbaijani. Thanks to its compact 322M parameter footprint, it can even run locally on a smartphone with a compatible inference runtime.

Free API and Fully Open Source

HAL-X AI has taken a developer-friendly approach to deployment. THX-01's API offers a free tier of 200 decisions per minute, letting developers experiment without worrying about infrastructure costs.

A typical API call is straightforward — simply pass in a context string and a typed question. For example, determining whether "my card was charged twice for one order" should go to the billing, technical, or sales team returns a classification result with associated probabilities.

The entire project is open-sourced under the Apache 2.0 license, with model weights, source code, and training resources all publicly available on Hugging Face. The Python package can be installed directly:

pip install thx01

The model and an interactive demo are both hosted on Hugging Face, allowing developers to quickly validate performance on their own use cases.

Is a Small, Specialized Decision Model Worth Paying Attention To?

HAL-X AI has set itself a clear mission: to build AI models that "know how to make decisions, know how confident they should be, and know when they don't know." This actually points to a real weakness in current generative AI — large models are good at generating fluent text, but often struggle to express their uncertainty in a trustworthy way.

Based on what's publicly available, THX-01's positioning is clear: it's not another general-purpose large model, but a specialized engine for high-frequency decision tasks like classification, extraction, and judgment. In an era of increasingly complex agent architectures with growing sensitivity to latency and cost, "small and specialized" components like this have genuine practical value.

Of course, 70 million tokens processed and strong internal benchmark numbers are largely self-reported, and the latency comparison has acknowledged environmental inconsistencies. The real test will come from independent community reproduction on diverse real-world datasets. That said, a fully open-source, free-to-try decision model that runs on a phone represents a low-barrier experimentation option for developers.

In AI agent system architectures, the "routing layer" or "judgment layer" component that THX-01 represents is attracting increasing attention. Complex agent pipelines typically involve multiple models working in concert: lightweight decision models handle rapid classification, confidence assessment, and task routing, escalating requests to more expensive large models only when confidence is insufficient. This cascaded inference architecture can dramatically reduce overall inference cost and latency while maintaining quality. THX-01 explicitly positions itself for this scenario — its "knowing when it doesn't know" design philosophy is precisely intended to enable agent systems to make rational routing decisions based on trustworthy confidence signals, rather than blindly relying on a single large model for every request.

Background: A Note on ECE

Expected Calibration Error (ECE) is the standard metric for measuring alignment between a model's confidence scores and its actual accuracy. Ideally, when a model says "I'm 80% confident," roughly 80% of those predictions should actually be correct — that's what "well-calibrated" means. Lower ECE means the model's probability outputs are more trustworthy. Standard training approaches (like cross-entropy loss) only penalize incorrect predictions without constraining calibration quality, causing many models to be either overconfident or underconfident. RLCD incorporates calibration quality into the optimization objective through reinforcement learning reward design, ensuring that THX-01's probability outputs have genuine reference value — not just a softmax afterthought attached to the classification result.