Information Theory: Principles and Applications

Information Theory: Principles and Applications

Professors: Tiago Travassos Vieira Vinhoza, Paulo J. S. G. Ferreira, and José Vieira
Class Hours: Fridays from 14:00 to 18:00 (FEUP, Room B024)
Office Hours: Wednesdays from 14:30 to 15:30 (FEUP, Room I322)

Notes:

22 Mar - Slides for Lecture 1 posted.
22 Mar - Some interesting papers related to our first lecture posted
28 Mar - Homework 2 posted.
28 Mar - Slides for Lecture 2 posted.
09 Apr - Homework 3 posted
09 Apr - Slides for Lecture 3 posted
09 Apr - Some interesting papers related to our third lecture posted
16 Apr - Homework 4 posted
16 Apr - Slides for Lecture 4 posted
16 Apr - Some interesting papers related to our fourth lecture posted
20 Apr - Two papers posted: Blahut algorithm to compute channel capacity and The road to channel capacity.
26 May - Slides from Prof. Paulo Ferreira classes posted
13 Jun - First set of slides from Prof. José Vieira posted
22 Jun - Second set of slides from Prof. José Vieira posted
02 Jul - Slides from Lectures 8 and 9 posted

Lecture Slides and other material:

22 Mar - Lecture 1 (PDF)
22 Mar - Shannon's 1948 paper (PDF)
22 Mar - Shannon "The Bandwagon" paper (PDF)
22 Mar - Wiener "What is Information Theory?" paper (PDF)
28 Mar - Lecture 2 (PDF)
28 Mar - Sardinas-Patterson test for unique decodability (here)
09 Apr - Lecture 3 (PDF)
09 Apr - Arithmetic Coding for Data Compression (PDF)
09 Apr - Arithmetic Coding Revealed (PDF)
16 Apr - Lecture 4 (PDF)
16 Apr - The ABCs of Linear Block Codes (PDF)
20 Apr - Channel Coding: The Road to Channel Capacity (PDF)
20 Apr - Computation of Channel Capacity and Rate-Distortion Functions (PDF)
26 May - Lecture 5 (PDF)
26 May - Lecture 6 (PDF)
26 May - Lecture 7 (PDF)
13 Jun - Lecture 10 (PDF)
22 Jun - Lecture 11 (PPT)
02 Jul - Lecture 8 (PDF)
02 Jul - Lecture 9 (PDF)
Please contact me if you spot any typo in the slides. Thanks.

Homework:

Homework 1 (PDF) - due date: 26 Mar.
Homework 2 (PDF) - due date: 16 Apr.
Homework 3 (PDF) - due date: 23 Apr.
Homework 4 (PDF) - due date: 30 Apr.

Homework evaluation
- Before due date: 100%
- Up to 1 week after due date: 80%
- Up to 2 weeks after due date: 60%
- More than 2 weeks after due date: zero

References:

Class Slides
Elements of Information Theory, Cover and Thomas, Wiley
Information Theory and Reliable Communications, R. G. Gallager, Wiley, 1968
Information Theory, Inference, and Learning Algorithms , David McKay (available online here)
Probability, Random Variables and Stochastic Processes, A. Papoulis, McGraw-Hill
Discrete Probability Refresher, F. Kschischang (available online here)
(back)

On 02 Jul 2010, 13:30.