Company
InferenceKey
A team focused on large model inference deployment and performance measurement that integrates various inference optimization components and conducts empirical evaluations.
Timeline (last 90 days)
Oct 16
The working solution relies on a combination: DeepSeek provides the weights, coolbho3k does the 3-bit quantization, turboderp wrote the EXL3 format and the ExLlamaV3 inference library, Victor Cruz wrote the vllm-exl3 bridge layer, Diffbot designed the per-expert split scheme, startup flow and kernels, and InferenceKey did the integration and measurement.
Unverified50%