Causal Encoder-Decoder
A hybrid Transformer architecture design that divides model layers into two components: a causal encoder and a decoder. The encoder processes only input tokens, while the decoder is used exclusively for output prediction. This separation reduces overall computational load and significantly compresses KV cache memory usage.
Timeline (last 90 days)
DeepSeek V4.1 Flash adopts a Causal Encoder-Decoder (CED) architecture, in which a 40-layer transformer has the first 20 layers as a causal encoder and the last 20 layers as a decoder.
In the CED architecture, the encoder runs only when processing input tokens and the decoder runs only when predicting output, meaning only half of the model weights need to be activated during the input phase.
In the CED architecture, the decoder's global KV cache is projected from the hidden states of the encoder's last layer, rather than from the hidden states of each individual layer.
All Facts (3)
DeepSeek V4.1 Flash adopts a Causal Encoder-Decoder (CED) architecture, in which a 40-layer transformer has the first 20 layers as a causal encoder and the last 20 layers as a decoder.
50%UnverifiedIn the CED architecture, the encoder runs only when processing input tokens and the decoder runs only when predicting output, meaning only half of the model weights need to be activated during the input phase.
50%UnverifiedIn the CED architecture, the decoder's global KV cache is projected from the hidden states of the encoder's last layer, rather than from the hidden states of each individual layer.
50%