Teaching

Deep Learning - Theorie und Praxis

Instructors: Univ.-Prof. Dr. Michael Wand
Shortname: 08.079.318
Course No.: 08.079.318
Course Type: Vorlesung/Übung

Requirements / organisational issues

The lecture builds on top of "Modelling 1" but a good math background (from minor or major studies in math or physics) is usually also sufficient. Prior knowledge in machine learning is useful, but not strictly required (there wil be a short recap in the beginning).

Contents

The course discusses the theory of deep neural networks. While we still do not fully understand why they work so well, some seemingly magical properties and observations have by now been demystified, often using surprisingly basic mathematical tools such as basic linear algebra or low-order Taylor-series approximations of non-linear network.

The course is intended as an introduction to methods and results for/from analyzing deep networks. The focus is on theory, not practice (a practice-focused course is offered as an alternative, also in this WS24/25).

The course is an update / extension of the earlier "Modelling 2 - statistical data modeling".

Tentative list of (potential) topics:

1 Basics

  1. Machine learning recap
  2. Information theory
  3. Generalization and Bayesian Inference
  4. Basic Deep Learning Methods

2 Mathematical tools

  1. Statistics in high-dimensional spaces
  2. Matrix factorization, Embeddings, Kernels
  3. Gaussian Processes
  4. Markov-Random Fields
  5. Manifolds and non-Euclidean metrics

3 Understanding Deep Networks

  1. Linear approximation via Gaussian processes (Neural Tangent Kernel etc.)
  2. "Double Descent" - generalization of overparametrized models
  3. How optimization algorithms affect learning and generalization (in unexpected ways)
  4. Models based on differential geometry, e.g. Fischer-information
  5. Ideas from statistical physics (symmetry, renormalization, phase transitions)

The lecture is designed for master students, but participants from the B.Sc. programme are welcome as well. It can be taught in Englisch and German (tbd. in the first lecture in discussion with the participants).

Additional information

Further information:

https://luna.informatik.uni-mainz.de/dl-24-25/

Dates

Date (Day of the week) Time Location
10/21/2024 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
10/28/2024 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
11/04/2024 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
11/11/2024 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
11/18/2024 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
11/25/2024 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
12/02/2024 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
12/09/2024 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
12/16/2024 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
01/06/2025 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
01/13/2025 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
01/20/2025 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
01/27/2025 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik
02/03/2025 (Monday) 12:15 - 13:45 04 432
2413 - Neubau Physik/Mathematik