Welcome to the page of "Information Theory (INFT530105)"

Course Meeting Time:

• Letures: 2 sessions/week, 2 class hours/session

Course Information:

• This is a undergraduate-level introduction to information theory. It offers an introduction to the quantitative theory of information and its applications to reliable, efficient communication systems. Topics include mathematical definition and properties of information, source coding theorem, lossless compression of data, optimal lossless coding, channel coding theorem, etc.

Reading Materials:

Textbook

• Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. 2nd ed. New York, NY: Wiley-Interscience, 2006. ISBN: 9780471241959.

Recommended Reading

• Robert G. Gallagher. Information Theory and Reliable Communication. New York, NY: Wiley, 1968. ISBN: 9780471290483.
• David J. C. MacKay. Information Theory, Inference, and Learning Algorithms. Cambridge University Press, 2003. ISBN: 9780521642989.
• Abbas El Gamal and Young-Han Kim. Network Information Theory. Cambridge University Press, 2011. ISBN: 9781107008731.
• Sergio Verdu, "Fifty years of Shannon theory," IEEE Transactions on Information Theory, vol. 44, no. 6, pp. 2057-2078, Oct. 1998.
• C. E. Shannon, "A mathematical theory of communication," The Bell System Technical Journal, vol. 27, pp. 379-423, 623-656, July, Oct. 1948. (Birth of Information Theory)

Course Topics:

• Chapter 1 -- Introduction to the content (slides)
• Chapter 2 -- Entropy and mutual inforamtion (slides)
• Chapter 3 -- Asymptotic equipartition property (slides)
• Chapter 4 -- Data compression (Source coding) (slides)
• Chapter 5 -- Channel capacity and channel coding theorem (slides)
• Chapter 6 -- Differential entropy and Guassian channel (slides)
• Comprehensive Review (slides)

Homeworks:

• Homework for Chapter 2 (download)
• Homework for Chapter 4 (download)
• Homework for Chapter 5 (download)