L3 Generalization, regularization, model selection and validation
Lecture slides (Lecture 3 Download Lecture 3 + L3 with content on Bayesian regularisation Download L3 with content on Bayesian regularisation)
Date and venue: Sep 5, 10:15-12:00, Lecture hall D2
Sep 9, 13:15-15:00, Lecture hall D2 (half of the session)
Session format: Q&A discussions dedicated to generalisation, regularisation (including Bayesian regularisation) in the spirit of a flipped classroom
Before the lecture |
Short introduction to lecture on generalisation and regularisation (L3): video and slides Download slides |
Pre-recorded video on regularisation and another one focused on Bayesian regularisation. |
Discussion during the online session: L3 Download L3
After the lecture |
Lecture 3 material: Lect3 video |
Scope, topics covered:
- generalisation, bias-variance dilemma
- cross-validation, bootstrapping,
- model selection, evaluation
- early stopping, network growing/pruning
- penalty term regularisation
- Bayesian regularisation
- ensemble learning
- dropout regularisation
Reading material:
- S. Marsland: 3.3, 8.1, 8.2 (2nd ed.: 2.3-2.5, 4.3)
- R. Rojas: 9.1-9.2, 8.5
- I. Goodfellow, Y. Bengio, A. Courville: 5.1-5.4, 7.1-7.8
- I. Goodfellow, Y. Bengio, A. Courville: 7.11 (ensembles)