COM-621 / 2 credits
Remark: Not given this year, next time: Spring 2025
Only this year
The class will focus on information-theoretic progress of the last decade. Topics include: Network Information Theory ; Information Measures: definitions, properties, and applications to probabilistic models.
1) Information measures: Definitions, properties, applications, pitfalls.
- Mutual information
- Directed information
- Wyner's common information
- Entropy-power inequality
- Renyi- and f-divergences
- Extremization of information measures
2) Information measures in probabilistic systems
- Generalization guarantees for learning algorithms
- Compressed Sensing
3) Network Information Theory
- Classical channel settings: Multiple-Access, Broadcast, Relay
- Classical source settings: Slepian-Wolf, Lossy source coding, CEO problem
- "Gaussian location" model and problem
- Application to federated learning?
COM-404 Information Theory and Coding
By the end of the course, the student must be able to:
- Generalize information measures
- Formulate estimation, inference and decision problems via the lens of information measures
- Analyze communication networks via information measures
- Manipulate information measures
Students will work on projects.
In the programs
- Exam form: Project report (session free)
- Subject examined: Advanced Topics in Information Theory
- Lecture: 14 Hour(s)
- Exercises: 14 Hour(s)