Unverified50% confidenceFactExact time
信息论先驱香农指出任何信道容量都存在上限，当信源信息熵超过信道容量时信息损失不可避免
1
Sources
50%
Confidence
Long-term
Relevance
7/7/2026
First Seen
Sources
Why Large Language Models Are Starting to 'Invent Words': The Nature of LLM Language Boundaries
hackernewshackernews7/6/2026
Related Claims
Unverified从信息论和香农定理角度看，任何超过信息熵下界的压缩都必然丢失信息，因此无论模型多大，压缩必然是有损的78% similarUnverified香农提出的信息熵概念表明低概率事件携带更高的信息量76% similarUnverified信息瓶颈理论由Tishby等人于2000年提出，主张深度网络在压缩输入信息的同时应保留与任务相关的最小充分统计量68% similarUnverifiedThe choice of bottleneck dimensionality is essentially a trade-off between information compression rate and reconstruction quality, corresponding to rate-distortion theory; Shannon proved in 1959 that there exists a theoretical minimum coding rate lower bound for a given distortion tolerance.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/168698API
curl https://kongchang.com/api/v1/knowledge/claims/168698MCP
get_claim(id=168698)