L7 Hopfield networks, associative memories and introduction to stochastic networks (Boltzmann machines)

Lecture slides: Lect7a Download Lect7a (Hopfield networks and associative memories), Lect7b Download Lect7b (Boltzmann machine)

Date and venue:   L7a + L7b on Sep 19, kl.13:15-15:00, Lecture hall K1                                                     

                                                           Sep 26, 8:15-10:00, Lecture hall D2

                                                           Sep 27, 10:15-12:00, Lecture hall D2

Session format:  lecture

Before the lecture

currently no pre-recorded video material

 

After the lecture

Lecture 7 material: Lect7 video (combined 7a+b)

 

Scope, topics covered

• Stochastic neural networks
• Hopfield networks
• Associative memory and attractors
• Boltzmann machines

Reading material:

• I. Goodfellow et al.: ch. 20.1 -2 
• R. Rojas: ch.12, 13, 14
• S. Haykin (2009, third ed.): section 11.6-8, 13.1-13.7
• Extra reading about Boltzmann machines and restricted Boltzmann machines (for those interested in more reading). See in particular points 2-4. Links to an external site. 
• Derivation of storage capacity, C~0.138N (from Hertz, Krogh, Palmer, Introduction to the theory of neural computation Download from Hertz, Krogh, Palmer, Introduction to the theory of neural computation
  ).