[KongchangAI]
Product

Hibana

An open-source AI inference engine designed for unified-memory devices, jointly optimizing through model surgery, unified memory management, and adaptive speculative decoding, supporting local multi-agent workloads.

Timeline (last 90 days)

Nov 10

Hibana is an inference engine designed specifically for unified-memory devices, jointly optimizing model representation, memory management and decoding strategy.

Unverified50%
Nov 10

Hibana adopts Model Surgery, i.e., editing the architecture of an already-trained model to adapt it to the hardware, thereby reducing memory footprint and data movement, rather than simple quantization or pruning.

Unverified50%
Nov 10

Hibana's unified memory management can preserve each agent's state across turns, allowing agents to resume instantly without recomputing state.

Unverified50%
Nov 10

Hibana's Adaptive Speculation technology lets the model generate multiple tokens per step and dynamically adjust according to load and hardware state, maximizing throughput under limited bandwidth.

Unverified50%
Nov 10

When serving Qwen and Nemotron models with up to 125B parameters on a single DGX Spark, Hibana completes up to 3.4 times more agent tasks per hour and achieves up to 3.9 times higher decoding throughput than vLLM, NVIDIA Flare, llama.cpp and TensorFold.

Unverified50%
Nov 10

Hibana's code has been open-sourced on GitHub at https://github.com/sharar-project/hibana.

Unverified50%
Nov 10

DGX Spark is only one form of unified-memory device; the bandwidth and capacity of devices such as laptops and phones differ even more, so Hibana's speedup cannot simply be extrapolated to other device forms.

Unverified50%
Nov 10

Hibana has currently only released test results on DGX Spark, and its performance on other device forms such as laptops and phones remains to be verified.

Unverified50%

All Facts (8)

Unverified

Hibana has currently only released test results on DGX Spark, and its performance on other device forms such as laptops and phones remains to be verified.

50%
Unverified

Hibana adopts Model Surgery, i.e., editing the architecture of an already-trained model to adapt it to the hardware, thereby reducing memory footprint and data movement, rather than simple quantization or pruning.

50%
Unverified

Hibana's unified memory management can preserve each agent's state across turns, allowing agents to resume instantly without recomputing state.

50%
Unverified

Hibana's Adaptive Speculation technology lets the model generate multiple tokens per step and dynamically adjust according to load and hardware state, maximizing throughput under limited bandwidth.

50%
Unverified

When serving Qwen and Nemotron models with up to 125B parameters on a single DGX Spark, Hibana completes up to 3.4 times more agent tasks per hour and achieves up to 3.9 times higher decoding throughput than vLLM, NVIDIA Flare, llama.cpp and TensorFold.

50%
Unverified

Hibana is an inference engine designed specifically for unified-memory devices, jointly optimizing model representation, memory management and decoding strategy.

50%
Unverified

DGX Spark is only one form of unified-memory device; the bandwidth and capacity of devices such as laptops and phones differ even more, so Hibana's speedup cannot simply be extrapolated to other device forms.

50%
Unverified

Hibana's code has been open-sourced on GitHub at https://github.com/sharar-project/hibana.

50%

Source Articles