Victor Cruz
An open-source developer and the author of the vllm-exl3 bridge layer, which integrates ExLlamaV3 quantization kernels into the vLLM inference framework.
Timeline (last 90 days)
The working solution relies on a combination: DeepSeek provides the weights, coolbho3k does the 3-bit quantization, turboderp wrote the EXL3 format and the ExLlamaV3 inference library, Victor Cruz wrote the vllm-exl3 bridge layer, Diffbot designed the per-expert split scheme, startup flow and kernels, and InferenceKey did the integration and measurement.
EXL3 is a 3-bit quantization format introduced by ExLlamaV3 and is the successor to EXL2; ExLlamaV3 is an inference library maintained by turboderp; vllm-exl3 is a bridge layer written by Victor Cruz that connects ExLlamaV3's quantization kernels into vLLM's scheduling and PagedAttention system.
All Facts (2)
The working solution relies on a combination: DeepSeek provides the weights, coolbho3k does the 3-bit quantization, turboderp wrote the EXL3 format and the ExLlamaV3 inference library, Victor Cruz wrote the vllm-exl3 bridge layer, Diffbot designed the per-expert split scheme, startup flow and kernels, and InferenceKey did the integration and measurement.
50%UnverifiedEXL3 is a 3-bit quantization format introduced by ExLlamaV3 and is the successor to EXL2; ExLlamaV3 is an inference library maintained by turboderp; vllm-exl3 is a bridge layer written by Victor Cruz that connects ExLlamaV3's quantization kernels into vLLM's scheduling and PagedAttention system.
50%