IQuest-Q1
A 320B-parameter Mixture-of-Experts (MoE) model developed by the IQuest_research team, designed for intelligent programming scenarios. It supports an ultra-long context window of 512K tokens and activates only 15B parameters per token through a sparse activation architecture.
Timeline (last 90 days)
IQuest-Q1 is a 320B-parameter Mixture of Experts (MoE) model introduced by the IQuest_research team, specifically designed for agentic coding scenarios.
IQuest-Q1 has 320 billion total parameters, with only 15 billion parameters activated per token.
IQuest-Q1 is equipped with 256 experts, of which only 8 are active during each inference.
IQuest-Q1 supports a context window of up to 524,288 tokens (512K).
IQuest-Q1 employs a mixed ratio of 3 layers of sliding window attention combined with 1 layer of full attention, with only 25 out of 88 layers expanding the KV cache as context grows.
IQuest-Q1 allows approximately 28% of its layers to participate in global cache growth, significantly reducing memory pressure.
IQuest-Q1's checkpoint requires the use of sink attention, and vLLM's attention backend provides a corresponding sinks path to support this.
IQuest-Q1 leverages EAGLE speculative decoding in vLLM combined with probabilistic draft sampling, with draft generation completed by a recursive MTP (Multi-Token Prediction) head.
IQuest-Q1's day-0 support stems from designing the model architecture around three existing capabilities in vLLM—mixed KV caching, sink attention path, and EAGLE speculative decoding.
All Facts (9)
IQuest-Q1 is a 320B-parameter Mixture of Experts (MoE) model introduced by the IQuest_research team, specifically designed for agentic coding scenarios.
50%UnverifiedIQuest-Q1 has 320 billion total parameters, with only 15 billion parameters activated per token.
50%UnverifiedIQuest-Q1 is equipped with 256 experts, of which only 8 are active during each inference.
50%UnverifiedIQuest-Q1 supports a context window of up to 524,288 tokens (512K).
50%UnverifiedIQuest-Q1 employs a mixed ratio of 3 layers of sliding window attention combined with 1 layer of full attention, with only 25 out of 88 layers expanding the KV cache as context grows.
50%UnverifiedIQuest-Q1 allows approximately 28% of its layers to participate in global cache growth, significantly reducing memory pressure.
50%UnverifiedIQuest-Q1's checkpoint requires the use of sink attention, and vLLM's attention backend provides a corresponding sinks path to support this.
50%UnverifiedIQuest-Q1 leverages EAGLE speculative decoding in vLLM combined with probabilistic draft sampling, with draft generation completed by a recursive MTP (Multi-Token Prediction) head.
50%UnverifiedIQuest-Q1's day-0 support stems from designing the model architecture around three existing capabilities in vLLM—mixed KV caching, sink attention path, and EAGLE speculative decoding.
50%