Information theory answers two fundamental questions in communication theory: What is
the ultimate data compression (answer: the entropy H), and what is the ultimate transmission rate
of communication (answer: the channel capacity C). For this reason, some consider information
theory to be a subset of communication theory. Indeed, it has fundamental contributions to make in
statistical physics (thermodynamics), computer science (Kolmogorov complexity or algorithmic
complexity), statistical inference (Occam’s Razor: “The simplest explanation is best”), and to
probability and statistics (error exponents for optimal hypothesis testing and estimation).
Copyrights © 2026 Department Of Statistics -CUSAT | All rights reserved | Disclaimer | Powered by CIRM