ProductNori LLM

Nori

An open-source LLM inference engine project focused on optimizing large language model inference performance. It pursues high throughput through techniques such as continuous batching and PagedAttention, positioning itself as a performance-oriented runtime for serving large language models.

Timeline (last 90 days)

Sep 24

An LLM inference project called Nori claims to have achieved a processing throughput of over 1 million tokens per second (1M tok/s).

Unverified50%

All Facts (1)

Unverified

An LLM inference project called Nori claims to have achieved a processing throughput of over 1 million tokens per second (1M tok/s).

50%

Source Articles

Nori LLM Breaks the Million Tokens-Per-Second Barrier: Pushing the Limits of Inference Engine PerformanceSep 24