MoR-MLLM
MoR-MLLM is a multimodal large language model built on the Mixture-of-Recursions framework. It dynamically adjusts computation depth through an adaptive per-token recursion mechanism, reducing training memory and computational complexity while maintaining high performance on multimodal tasks.
Timeline (last 90 days)
A paper published on arXiv proposes MoR-MLLM, which reconstructs the computational paradigm of multimodal models by adaptively allocating compute per token.
MoR-MLLM is built on the Mixture-of-Recursions (MoR) framework.
MoR-MLLM introduces an adaptive per-token recursion mechanism that allocates more recursive computation to more challenging tokens and skips redundant computation for simple or redundant tokens.
The paper proposes a three-stage MoR-Tuning procedure that progressively guides the model to perform reasonable recursive sparsity on multimodal inputs.
The paper introduces an entropy-regularized loss to prevent the routing distribution from becoming overly concentrated, encouraging a more diverse routing distribution.
Experiments show that compared to recent state-of-the-art small MLLMs, MoR-MLLM can significantly reduce training memory and computational complexity while maintaining high performance on a variety of vision-language tasks.
As a newly released research paper, MoR-MLLM's generalization capability and performance on larger-scale models still await further validation.
All Facts (7)
The paper proposes a three-stage MoR-Tuning procedure that progressively guides the model to perform reasonable recursive sparsity on multimodal inputs.
50%UnverifiedMoR-MLLM is built on the Mixture-of-Recursions (MoR) framework.
50%UnverifiedMoR-MLLM introduces an adaptive per-token recursion mechanism that allocates more recursive computation to more challenging tokens and skips redundant computation for simple or redundant tokens.
50%UnverifiedAs a newly released research paper, MoR-MLLM's generalization capability and performance on larger-scale models still await further validation.
50%UnverifiedThe paper introduces an entropy-regularized loss to prevent the routing distribution from becoming overly concentrated, encouraging a more diverse routing distribution.
50%UnverifiedExperiments show that compared to recent state-of-the-art small MLLMs, MoR-MLLM can significantly reduce training memory and computational complexity while maintaining high performance on a variety of vision-language tasks.
50%UnverifiedA paper published on arXiv proposes MoR-MLLM, which reconstructs the computational paradigm of multimodal models by adaptively allocating compute per token.
50%