Unverified60% confidenceFactExact time
Work by Shwartz-Ziv and Tishby in 2017 showed that deep network training has two phases: first fitting the data (increasing mutual information with the input), then compressing the representation (reducing mutual information with the input while preserving mutual information with the output).
2
Sources
60%
Confidence
Long-term
Relevance
8/8/2026
First Seen
Sources
Related Claims
UnverifiedInformation bottleneck theory was proposed by Tishby et al. in 1999 and developed in 2017 into an information-theoretic interpretive framework for deep learning, arguing that neural network training consists of two stages: rapid fitting and slow compression.77% similarUnverifiedThe information bottleneck hypothesis proposed by Shwartz-Ziv and Tishby attempts to explain why deep learning works from the perspective of information compression76% similarUnverified残差连接与层归一化解决了深层网络训练不稳定的问题，使模型可堆叠到数十层75% similarVerifiedDeep Ensembles train multiple independent networks and aggregate their predictive distributions, achieving the best performance but at the highest computational cost75% similarUnverified跳跃连接通过提供梯度直通路径解决了深度网络训练中的梯度消失问题，使极深网络训练成为可能73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/707890API
curl https://kongchang.com/api/v1/knowledge/claims/707890MCP
get_claim(id=707890)