Statistical physics for optimization & learning
PHYS-642 / 4 credits
Teacher(s): Krzakala Florent Gérard, Loureiro Bruno, Saglietti Luca, Zdeborová Lenka
Language: English
Remark: Next time: Spring 2027
Frequency
Every 2 years
Summary
This course covers the statistical physics approach to computer science problems, with an emphasis on heuristic & rigorous mathematical technics, ranging from graph theory and constraint satisfaction to inference to machine learning, neural networks and statitics.
Content
Powerful Mathematical techniques from statistical physics and spin glass theory have been applied with increasing success on various problems ranging from computer science, statistics to machine learning. In the last decades, in particular, there has been increasing convergence of interest and methods between theoretical physics and much theoretical and applied work in statistical physics and computer science has relied on the use of message-passing algorithms and their connection to the statistical physics of glasses and spin glasses.This course will cover this rich and active interdisciplinary research landscape.
A particular emphasis will be given to high-dimensional problems. Indeed modern data analysis uses complex statistical models with massive numbers of parameters In some cases, the high-dimensional limit is analogous to the "thermodynamic limit" of a certain (disordered) statistical mechanics system. Building on mathematical ideas from the mean-field theory of disordered systems, exact asymptotic can thus be computed for high-dimensional problems. We shall discuss examples in statistics, coding theory, and machine learning.
While the course is designed to be a follow up of PHYS-512, it is also intendeded to stand on its own, and to be accessible to mathematically-minded graduate students and researchers from engineering, computer science and mathematics disciplines with a knowledge of probability and analysis. The course is aimed at theory-minded students, interested in the use of powerful methods originating in statistical physics, and their connection to open problems in modern high-dimensional statistics, computer science and machine learning.
Note
Website of the lecture: https://idephics.github.io/EPFLDoctoralLecture2023/
Mainly a theoretical course, with exercises in the analytical methods and usage of the related algorithms in high-dimensional problems in statistics, optimization and machine learning
Evaluation of the lecture based on homeworks given during the whole semester
Keywords
Statistical physics, replica method, cavity method, neural networks, theory of machine learning, combinatorial optimization, community detection, graphical models, message passins algorithms.
Learning Prerequisites
Required courses
For physicists : PHYS 512 & a good knowlegde of statistical physics.
For mathematicians: Probability & Introductory statistical physics will be helpful
FOR CS/STI: Basic probability & Information theory/Entropy/Coding will be helpful
Learning Outcomes
By the end of the course, the student must be able to:
- To study a range of problems in computer science and learning, and derive formulas and algorithms for their solution, using technics from statistical physics.
- To study a range of problems in computer science and learning
- derive formulas and algorithms for their solution, using technics from statistical physics
In the programs
- Exam form: During the semester (session free)
- Subject examined: Statistical physics for optimization & learning
- Lecture: 28 Hour(s)
- Exercises: 28 Hour(s)
- Type: optional
- Exam form: During the semester (session free)
- Subject examined: Statistical physics for optimization & learning
- Lecture: 28 Hour(s)
- Exercises: 28 Hour(s)
- Type: optional