Kolibri-1
An open-source Mixture-of-Experts (MoE) large language model released by Aleph Alpha, featuring 78 billion total parameters with 3.46 billion parameters activated during inference. The model supports a context window of up to one million tokens and is distributed under the Apache 2.0 license.
Timeline (last 90 days)
This experiment is currently only a proof-of-concept small-scale experiment and cannot demonstrate Kolibri-1's comprehensive capabilities on complex tasks.
Kolibri-1 is an open-source model from Aleph Alpha, released under an open-weights strategy.
A developer had Kolibri-1 directly play the classic arcade game Breakout without any targeted fine-tuning.
In this experiment, the inference latency for each decision step of Kolibri-1 was kept within 25 milliseconds.
A single-step latency of 25 milliseconds means the model can theoretically make approximately 40 decisions per second.
This experiment directly converted the model's output action probabilities into game operations, rather than generating text.
Directly reading the probability values from the logits layer, bypassing the redundant text generation step, is the key to achieving the low latency of 25 milliseconds.
This experiment emphasizes direct reasoning under zero-shot or minimal constraints, rather than training a game-specific model.
Aleph Alpha released a new model, Kolibri-1, on Hugging Face.
Kolibri-1 is open-sourced under the Apache 2.0 license.
6 more timeline events
All Facts (16)
This experiment is currently only a proof-of-concept small-scale experiment and cannot demonstrate Kolibri-1's comprehensive capabilities on complex tasks.
50%UnverifiedKolibri-1 is an open-source model from Aleph Alpha, released under an open-weights strategy.
50%UnverifiedIn this experiment, the inference latency for each decision step of Kolibri-1 was kept within 25 milliseconds.
50%UnverifiedA single-step latency of 25 milliseconds means the model can theoretically make approximately 40 decisions per second.
50%UnverifiedThis experiment directly converted the model's output action probabilities into game operations, rather than generating text.
50%UnverifiedDirectly reading the probability values from the logits layer, bypassing the redundant text generation step, is the key to achieving the low latency of 25 milliseconds.
50%UnverifiedThis experiment emphasizes direct reasoning under zero-shot or minimal constraints, rather than training a game-specific model.
50%UnverifiedSparse activation reduces computational load rather than memory footprint; the total 78B parameters still require considerable VRAM or memory resources.
50%UnverifiedKolibri-1 supports a context length of up to 1 million tokens.
50%UnverifiedAleph Alpha has provided a technical report for Kolibri-1 explaining the model's training and architectural details.
50%UnverifiedKolibri-1 is open-sourced under the Apache 2.0 license.
50%UnverifiedKolibri-1 adopts a Mixture of Experts (MoE) architecture.
50%UnverifiedKolibri-1 has a total parameter count of 78B, with only approximately 3.46B parameters activated during inference.
50%UnverifiedKolibri-1's activation ratio is less than 5%.
50%UnverifiedA developer had Kolibri-1 directly play the classic arcade game Breakout without any targeted fine-tuning.
50%UnverifiedAleph Alpha released a new model, Kolibri-1, on Hugging Face.
50%