MMoE
A multi-task learning architecture that places multiple expert subnetworks in the shared layer, with each task using an independent gating network to weight and combine the expert outputs, thereby alleviating task conflicts while retaining the benefits of parameter sharing.
Timeline (last 90 days)
MMoE (Multi-gate Mixture-of-Experts) was proposed by Google in 2018, and its core is placing multiple expert sub-networks in the shared layer, with each task having an independent gating network that weights and combines the expert outputs.
Compared with Hard Parameter Sharing, MMoE usually performs more stably when multi-task correlations are weak, but its structure is more complex, its training cost is higher, and the gating weights require enough data to be learned well.
All Facts (2)
MMoE (Multi-gate Mixture-of-Experts) was proposed by Google in 2018, and its core is placing multiple expert sub-networks in the shared layer, with each task having an independent gating network that weights and combines the expert outputs.
50%UnverifiedCompared with Hard Parameter Sharing, MMoE usually performs more stably when multi-task correlations are weak, but its structure is more complex, its training cost is higher, and the gating weights require enough data to be learned well.
50%