Information Theory Basics = Elements of Information Theory: Elements of Information Theory

Author: Thomas M. Cover
Publisher:
Publish Date: 2003-11-01
Features: This book systematically introduces the fundamental principles of information theory and its applications in fields such as communication theory, statistics, computer science, probability theory, and investment theory. The author presents the material in a progressive manner, introducing the basic definitions of information quantity, relative entropy, mutual information, and how they naturally apply to solving problems such as data compression, channel capacity, information rate distortion, statistical hypotheses, and network information flow. In addition, the book explores many issues rarely covered in textbooks, such as the connection between the second law of thermodynamics and Markov chains, the optimality of Huffman coding, the duality of data compression, Lempel-Ziv coding, Kolmogorov complexity, Portfolio theory, information theory inequalities, and their mathematical conclusions. This book can serve as a textbook or reference for senior undergraduate and graduate students in communication, electronics, computer science, automatic control, statistics, and economics, as well as for researchers and professionals in related fields.