Unverified70% confidenceFactExact time
Information bottleneck theory was proposed by Tishby et al. in 1999 and developed in 2017 into an information-theoretic interpretive framework for deep learning, arguing that neural network training consists of two stages: rapid fitting and slow compression.
2
Sources
70%
Confidence
Long-term
Relevance
7/16/2026
First Seen
Sources
Related Claims
UnverifiedWork by Shwartz-Ziv and Tishby in 2017 showed that deep network training has two phases: first fitting the data (increasing mutual information with the input), then compressing the representation (reducing mutual information with the input while preserving mutual information with the output).77% similarUnverifiedThe information bottleneck hypothesis proposed by Shwartz-Ziv and Tishby attempts to explain why deep learning works from the perspective of information compression76% similarUnverified深度学习中突破性的模型架构与训练技巧多为实践先行、理论解释滞后数年71% similarUnverifiedNeural networks encounter vanishing gradient problems as layers are stacked deeper, making training difficult to converge, and it was not until around 2010 that a series of technical breakthroughs solved the challenge of training deep networks, giving rise to the term deep neural network (DNN).70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/533286API
curl https://kongchang.com/api/v1/knowledge/claims/533286MCP
get_claim(id=533286)