Small Language Model (SLM)

Small Language Models (SLMs) generally refer to models with parameter counts between 1B and 7B.

Small language models struggle to spontaneously discover effective tool-calling behavior through reinforcement learning alone, because the search space is too vast and sampling efficiency is low.

For language rewriting tasks, small models paired with strict factual constraints are entirely usable.

Small models of 1B to 3B parameters cannot reliably plan loop routes on real street networks, inventing non-existent paths or producing routes that cross rivers.

The value proposition of entropy methods in SLMs is not computational savings, but intelligent allocation of computing resources.

Semantic entropy successfully recovers a usable confidence signal, filling the blind spot of token-level entropy on SLMs.

In small language models, confidence signals based on token entropy almost entirely fail to distinguish between correct and incorrect answers.

In 91% of dataset–model combinations, the average token entropy was close to zero regardless of whether the answer was correct or not.

Specialized models correspond to lower hypothesis space complexity and tighter generalization bounds; in data-scarce scenarios, a small amount of high-quality professional data combined with a highly aligned model architecture often outperforms large models trained on massive general-purpose data.

Using cheap small models to handle auxiliary tasks such as filtering, routing, and summarization, and then passing the refined information to an expensive main model, is a layered architectural approach to controlling cost and latency.

The inference cost of SLMs can be 1 to 2 orders of magnitude lower than mainstream large models, and they can run locally or on edge devices.

Parsec introduces a small language model (SLM) specifically trained for token compression, which preprocesses content before it enters the main model.

Interactive orchestration strategies (such as multi-model debate and iterative verification) require models to maintain multi-turn conversational context, placing high demands on context window length and computational resources—precisely the two main bottlenecks of SLMs

The non-interactive paradigm circumvents the bottleneck dependencies on SLM context windows and computation by fully decoupling generation from selection

The relatively limited capabilities and context windows of SLMs constrain their performance on long-chain reasoning and orchestration strategies that depend on high-frequency interactions

Recent research indicates that the large number of repetitive and narrowly scoped subtasks in agentic workloads may be better handled by specialized SLMs than by a single large LLM

Small language models (SLMs) typically have parameter counts within a few billion, aiming to run locally on devices such as phones and laptops.

With advances in quantization techniques and inference optimization, models in the 1B-3B parameter range are regaining attention, as they can run on laptops, Raspberry Pis, or even phones.