Laya AI
A non-autoregressive model specialized for decision-making, built on a BERT/M-BERT backbone that discriminatively scores candidate answers. A single forward pass outputs a decision with calibrated probabilities at a latency of approximately 35 milliseconds, with a model size of approximately 850 MB.
Timeline (last 90 days)
Laya AI is a non-autoregressive decision model that outputs answers with calibrated probabilities in a single forward pass, taking about 35 milliseconds.
Laya AI's model file (saved tensors) is about 850MB.
Laya AI can run on both CPU and GPU, and running on a 20-core CPU uses only about 30% of the CPU.
Laya AI uses a BERT/M-BERT backbone network to score candidate answers discriminatively, rather than generating text token by token.
Laya is installed by installing the library via pip, and on first run it automatically downloads the model from Hugging Face.
Laya supports discrete multiple-choice tasks, where the caller dynamically provides candidate labels at runtime, and the model scores each candidate independently and normalizes the scores into a probability distribution.
In the customer service ticket routing example, given a duplicate-charge complaint text, Laya outputs that it should be routed to the billing department, along with the probability and the language being English.
Laya's base checkpoint can be fine-tuned on custom domain data, and the fine-tuning data only needs "state-question-correct option" triples, without generative target text.
Laya excels at closed-ended questions (quickly scoring given a state and options) and is not suitable for open-ended generation, multi-turn dialogue, or complex reasoning.
Laya can complement traditional LLMs: the LLM handles understanding and generation, while Laya handles low-latency deterministic decisions.
All Facts (9)
Laya AI is a non-autoregressive decision model that outputs answers with calibrated probabilities in a single forward pass, taking about 35 milliseconds.
50%UnverifiedLaya AI's model file (saved tensors) is about 850MB.
50%UnverifiedLaya AI can run on both CPU and GPU, and running on a 20-core CPU uses only about 30% of the CPU.
50%UnverifiedLaya AI uses a BERT/M-BERT backbone network to score candidate answers discriminatively, rather than generating text token by token.
50%UnverifiedLaya is installed by installing the library via pip, and on first run it automatically downloads the model from Hugging Face.
50%UnverifiedLaya supports discrete multiple-choice tasks, where the caller dynamically provides candidate labels at runtime, and the model scores each candidate independently and normalizes the scores into a probability distribution.
50%UnverifiedLaya's base checkpoint can be fine-tuned on custom domain data, and the fine-tuning data only needs "state-question-correct option" triples, without generative target text.
50%UnverifiedLaya excels at closed-ended questions (quickly scoring given a state and options) and is not suitable for open-ended generation, multi-turn dialogue, or complex reasoning.
50%UnverifiedLaya can complement traditional LLMs: the LLM handles understanding and generation, while Laya handles low-latency deterministic decisions.
50%