[KongchangAI]
ConceptShannon entropy / 香农熵

Shannon Entropy

An information entropy measure proposed by Claude Shannon that quantifies the average uncertainty of a random variable. It is a foundational concept in information theory.

Timeline (last 90 days)

Nov 11

Thermodynamic entropy in physics and Shannon entropy in information theory are deeply connected in mathematical form, but they are not the same thing.

Unverified50%
Nov 9

As α→0, Renyi entropy approaches the observed vocabulary richness (the logarithm of the number of word types that have appeared); at α=1 it equals Shannon entropy; at α=2 it is called collision entropy; and as α→∞ it yields min-entropy.

Unverified50%
Oct 19

MaskFlow v2.0 uses a Shannon entropy threshold of H(X) ≥ 3.85 bits/char combined with character-class transitions to automatically identify and mask high-entropy unknown tokens.

Unverified50%
Oct 19

The high-entropy detection method does not rely on prefixes or fixed lengths and can cover random tokens generated by internal systems, but it may misclassify long hashes, compressed data blocks, or high-entropy test fixtures as secrets.

Unverified50%
Sep 22

Shannon entropy H(X) defines the limit achievable by lossless compression, i.e., the lower bound of information content.

Unverified50%

All Facts (5)

Source Articles