L3 Generalization, regularization, model selection and validation

Lecture slides (Lecture 3 Download Lecture 3 + L3 with content on Bayesian regularisation Download L3 with content on Bayesian regularisation

)

Date and venue: Jan 22, 15:15-17:00, Lecture hall B2                        

Session format: Q&A  discussions dedicated to generalisation, regularisation (including Bayesian regularisation) in the spirit of a flipped classroom

Before the lecture

Short introduction to lecture on generalisation and regularisation (L3): video and slides Download slides

Pre-recorded video on regularisation and another one focused on Bayesian regularisation.

Discussion during the online session: L3 Download L3

After the lecture

Lecture 3 material: Lect3 video and Lect3 slides Download Lect3 slides

 

Scope, topics covered:

• generalisation, bias-variance dilemma
• cross-validation, bootstrapping,
• model selection, evaluation
• early stopping, network growing/pruning
• penalty term regularisation
• Bayesian regularisation
• ensemble learning
• dropout regularisation

Reading material:

• S. Marsland: 3.3, 8.1, 8.2 (2nd ed.: 2.3-2.5, 4.3)
• R. Rojas: 9.1-9.2, 8.5 
• I. Goodfellow, Y. Bengio, A. Courville: 5.1-5.4, 7.1-7.8
• I. Goodfellow, Y. Bengio, A. Courville: 7.11 (ensembles)