AiJinTian Open-Source Demo Deng 3.5 74B
A large language model employing a sparse mixture-of-experts (MoE) architecture and two-bit quantization-aware training. It has a total of 74B parameters, activates about 9B parameters per pass, and its weight footprint can be compressed to within 24GB, supporting local deployment.
Timeline (last 90 days)
A project named '爱今天开源Demo灯3.5 74B' compresses a 74-billion-parameter model to 24GB of VRAM, exactly fitting a single RTX 4090 or 3090.
The project adopts a sparse mixture-of-experts (MoE) architecture, in which only about 9B parameters are activated per token out of 74B total parameters, an activation ratio of about 12%.
Running the project on a 12GB graphics card requires a more aggressive quantization scheme or a substantially shortened context.
Five mainstream runtimes provided support on the first day, and the weights went online on October 15 itself.
The project offers a free online version for users without high-end GPUs to try.
A few hours before the project's release, it still could not be found on HuggingFace.
The project also released the weights, training data, and training recipe.
There is currently no independent third-party evaluation; 'smarter than comparable models, faster than most dense models' is the team's self-assessment.
According to the team's official statement, local large model downloads have reached 5 times the whole of last year.
All Facts (9)
According to the team's official statement, local large model downloads have reached 5 times the whole of last year.
50%UnverifiedThe project adopts a sparse mixture-of-experts (MoE) architecture, in which only about 9B parameters are activated per token out of 74B total parameters, an activation ratio of about 12%.
50%UnverifiedRunning the project on a 12GB graphics card requires a more aggressive quantization scheme or a substantially shortened context.
50%UnverifiedThe project offers a free online version for users without high-end GPUs to try.
50%UnverifiedThe project also released the weights, training data, and training recipe.
50%UnverifiedThere is currently no independent third-party evaluation; 'smarter than comparable models, faster than most dense models' is the team's self-assessment.
50%UnverifiedFive mainstream runtimes provided support on the first day, and the weights went online on October 15 itself.
50%UnverifiedA project named '爱今天开源Demo灯3.5 74B' compresses a 74-billion-parameter model to 24GB of VRAM, exactly fitting a single RTX 4090 or 3090.
50%UnverifiedA few hours before the project's release, it still could not be found on HuggingFace.
50%