Information Theory


Welcome to the page of "Information Theory (INFT3044)"



Course Meeting Time:

  • Letures: 2 sessions/week, 2 class hours/session


Course Information:

  • This is a undergraduate-level introduction to information theory. It offers an introduction to the quantitative theory of information and its applications to reliable, efficient communication systems. Topics include mathematical definition and properties of information, source coding theorem, lossless compression of data, optimal lossless coding, channel coding theorem, etc.


Reading Materials:

  • Textbook
  • Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. 2nd ed. New York, NY: Wiley-Interscience, 2006. ISBN: 9780471241959.
  • Recommended Reading
  • Robert G. GallagherInformation Theory and Reliable Communication. New York, NY: Wiley, 1968. ISBN: 9780471290483.


Course Topics:

  • Chapter 1 -- Introduction to the content (slides)
  • Chapter 2 -- Entropy and mutual inforamtion (slides)
  • Chapter 3 -- Asymptotic equipartition property (slides)
  • Chapter 4 -- Data compression (Source coding) (slides)
  • Chapter 5 -- Channel capacity and channel coding theorem (slides)
  • Chapter 6 -- Differential entropy and Guassian channel (slides)
  • Comprehensive Review (slides)