Fundamentals of Information Theory

Author: Thomas M. Cover, Joy A. Thomas
Publisher:
Publish Date: 2003-11-01
Features: This book systematically introduces the fundamental principles of information theory and its applications in fields such as communication theory, statistics, computer science, probability theory, and investment theory. The authors present the material in a progressive manner, covering the basic definitions of information quantity, relative entropy, mutual information, and how they naturally solve problems like data compression, channel capacity, information rate distortion, statistical hypotheses, and network information flow. In addition, the book explores topics rarely covered in other textbooks, such as the connection between the second law of thermodynamics and Markov chains, the optimality of Huffman coding, the duality of data compression, Lempel-Ziv coding, Kolmogorov complexity, Portfolio theory, information theory inequalities, and their mathematical conclusions. This book can serve as a textbook or reference for senior undergraduate and graduate students in communication, electronics, computer science, automatic control, statistics, and economics, as well as for researchers and professionals in related fields.