LLMLingua
LLMLingua is a prompt compression method proposed by Microsoft Research that uses a small auxiliary language model to score the importance of individual tokens and selectively remove lower-importance ones, reducing inference costs for large language models.
Timeline (last 90 days)
LLMLingua uses a language model to identify and remove unimportant tokens, enabling 50% context length compression while maintaining 95% task performance.
Learning-based context compression frameworks such as LLMLingua can compress prompt length to one-fifth of the original or less while preserving semantics
All Facts (2)
LLMLingua uses a language model to identify and remove unimportant tokens, enabling 50% context length compression while maintaining 95% task performance.
50%UnverifiedLearning-based context compression frameworks such as LLMLingua can compress prompt length to one-fifth of the original or less while preserving semantics
50%