Data Processing Inequality
A classic theorem in information theory stating that any processing of data (summarization, compression, rewriting) cannot create new information about the original source; it can only lose information or leave it unchanged. In multi-agent pipelines, it explains why upstream summarization inevitably loses information that downstream stages cannot recover.
Timeline (last 90 days)
The data processing inequality guarantees that the push-forward measure from the data space distribution to the parameter space distribution does not increase the relative entropy, and the KL divergence between the parameter space posterior and prior is controlled by the KL divergence between the data space DP prior and posterior.
The data processing inequality states that if random variables X→Y→Z form a Markov chain, then the mutual information satisfies I(X;Z) ≤ I(Y;Z) ≤ I(X;Y); any processing applied to data (summarization, compression, rewriting) cannot create information about the original source out of thin air.
All Facts (2)
The data processing inequality guarantees that the push-forward measure from the data space distribution to the parameter space distribution does not increase the relative entropy, and the KL divergence between the parameter space posterior and prior is controlled by the KL divergence between the data space DP prior and posterior.
50%UnverifiedThe data processing inequality states that if random variables X→Y→Z form a Markov chain, then the mutual information satisfies I(X;Z) ≤ I(Y;Z) ≤ I(X;Y); any processing applied to data (summarization, compression, rewriting) cannot create information about the original source out of thin air.
50%