Topics in machine learning
MATH-520 / 5 crédits
Enseignant:
Langue: Anglais
Remark: Pas donné en 2024-25
Summary
Mathematical analysis of modern supervised machine learning techniques, from linear methods to artificial neural networks.
Content
-
Introduction (supervised learning, risk, error decomposition, over-fitting and capacity control + cross-validation, Bayes predictor for classification and regression)
-
Differentiable programming (backpropagation algorithm) and theoretical challenges posed by modern methods (large deep neural networks)
-
Statistical analysis of Empirical Risk Minimization (learning theory, from finite number of hypotheses to Rademacher / covering numbers)
-
First-order methods for optimization (gradient descent, stochastic gradient descent).
-
Kernel methods (positive-definite kernels and Reproducing Kernel Hilbert Spaces)
-
Algorithmic regularization of gradient descent (reparameterized models, least-squares, mirror descent, logistic loss and max-margin)
-
Dynamics of wide neural networks (parameterizations, neural tangent kernel and feature-learning limits)
Keywords
Supervised learning, Machine learning, Neural networks, Optimization, Statistics
Learning Prerequisites
Required courses
Analysis, Linear Algebra, Probability and Statistics
Important concepts to start the course
-
A good knowledge of undergraduate mathematics is important.
-
Ability to code in a scientific computing programming language of your choice (e.g. Python, Matlab, Julia). The course will involve coding exercises and assignments.
-
Having followed an introductory class on machine learning is beneficial.
Learning Outcomes
By the end of the course, the student must be able to:
- Contextualise the research literature on theoretical machine learning
- Interpret he practical behavior of complex machine learning pipelines through the lens of mathematical theory
- Implement simple supervised learning algorithms from scratch
- Reason on how statistical and optimization phenomena interact in machine learning practice
- Distinguish between what is known and what is not known in the theory of deep learning
Assessment methods
Homeworks, projects, presentation
Dans les plans d'études
- Semestre: Automne
- Forme de l'examen: Pendant le semestre (session d'hiver)
- Matière examinée: Topics in machine learning
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Pendant le semestre (session d'hiver)
- Matière examinée: Topics in machine learning
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Pendant le semestre (session d'hiver)
- Matière examinée: Topics in machine learning
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
- Semestre: Automne
- Forme de l'examen: Pendant le semestre (session d'hiver)
- Matière examinée: Topics in machine learning
- Cours: 2 Heure(s) hebdo x 14 semaines
- Exercices: 2 Heure(s) hebdo x 14 semaines
- Type: optionnel
Semaine de référence
Lu | Ma | Me | Je | Ve | |
8-9 | |||||
9-10 | |||||
10-11 | |||||
11-12 | |||||
12-13 | |||||
13-14 | |||||
14-15 | |||||
15-16 | |||||
16-17 | |||||
17-18 | |||||
18-19 | |||||
19-20 | |||||
20-21 | |||||
21-22 |
Légendes:
Cours
Exercice, TP
Projet, labo, autre