Antirion
An open-source AI gateway written in Rust that sits between applications and LLM providers, handling routing, fallback, rate limits, budgets, guardrails and logging, with a focus on very low overhead and high throughput.
Timeline (last 90 days)
Antirion is an open-source AI gateway built from scratch in Rust, billed as the world's fastest.
In a simple chat scenario at 5,000 req/s, Antirion's p50 overhead is 0.53 ms versus 19.7 ms for Bifrost 2.2.4, 37 times lower.
At 5,000 req/s, Antirion's peak memory is 215 MiB versus 1,389 MiB for Bifrost.
Antirion's maximum sustained throughput is 14,000 req/s versus 3,500 req/s for Bifrost.
At 10,000 req/s, Bifrost's median request wait time reaches 12 seconds, while Antirion's p99 stays within 2 ms.
Under a light load of 1,000 req/s, Bifrost's p50 is 0.72 ms and Antirion's is 0.76 ms, putting the two at similar performance.
Antirion uses a one-thread-per-CPU model, with each data thread running a single-threaded runtime and listening independently via SO_REUSEPORT, avoiding task migration across threads.
Antirion uses a custom JSON parser that records the offsets of JSON members in the raw bytes, so when the vendor and client dialects match, the request body is never copied (zero-copy).
Antirion performs a dry-run precheck of encoding before routing, excluding deployments that cannot serve the request up front and avoiding wasting a round trip on fallback.
Antirion's configuration uses immutable snapshots stored in an ArcSwap, loaded once at the start of each request, so configuration changes never take effect mid-request.
2 more timeline events
All Facts (12)
Antirion is an open-source AI gateway built from scratch in Rust, billed as the world's fastest.
50%UnverifiedIn a simple chat scenario at 5,000 req/s, Antirion's p50 overhead is 0.53 ms versus 19.7 ms for Bifrost 2.2.4, 37 times lower.
50%UnverifiedAt 5,000 req/s, Antirion's peak memory is 215 MiB versus 1,389 MiB for Bifrost.
50%UnverifiedAntirion's maximum sustained throughput is 14,000 req/s versus 3,500 req/s for Bifrost.
50%UnverifiedAt 10,000 req/s, Bifrost's median request wait time reaches 12 seconds, while Antirion's p99 stays within 2 ms.
50%UnverifiedUnder a light load of 1,000 req/s, Bifrost's p50 is 0.72 ms and Antirion's is 0.76 ms, putting the two at similar performance.
50%UnverifiedAntirion uses a one-thread-per-CPU model, with each data thread running a single-threaded runtime and listening independently via SO_REUSEPORT, avoiding task migration across threads.
50%UnverifiedAntirion uses a custom JSON parser that records the offsets of JSON members in the raw bytes, so when the vendor and client dialects match, the request body is never copied (zero-copy).
50%UnverifiedAntirion performs a dry-run precheck of encoding before routing, excluding deployments that cannot serve the request up front and avoiding wasting a round trip on fallback.
50%UnverifiedAntirion's configuration uses immutable snapshots stored in an ArcSwap, loaded once at the start of each request, so configuration changes never take effect mid-request.
50%UnverifiedAntirion's billing model reserves a token estimate at admission and settles exactly once with actual usage at request end; when a client disconnects midway the request is not cancelled and the partial cost already generated is still charged.
50%UnverifiedAntirion's code is divided into 8 layers totaling 28 crates, each crate may only depend on layers below it, and violating the rule fails the CI build.
50%