L7 Hopfield networks, associative memories and introduction to stochastic networks (Boltzmann machines)

Lecture slides: Lect7a Download Lect7a (Hopfield networks and associative memories), Lect7b Download Lect7b (Boltzmann machine)

Date and venue:   L7a + L7b on Feb 5, 15:15-17:00, Lecture hall B3                                                                                                                                                                     Feb 7, 13:15-15:00, Lecture hall D1

Session format:  lecture

Before the lecture

currently no pre-recorded video material

 

After the lecture

Lecture 7 material: Lect7a video (part 1, Feb 5), Lect7ab video (part 2, Feb 7)

 

Scope, topics covered

• Stochastic neural networks
• Hopfield networks
• Associative memory and attractors
• Boltzmann machines

Reading material:

• I. Goodfellow et al.: ch. 20.1 -2 
• R. Rojas: ch.12, 13, 14
• S. Haykin (2009, third ed.): section 11.6-8, 13.1-13.7
• Extra reading about Boltzmann machines and restricted Boltzmann machines (for those interested in more reading). See in particular points 2-4. Links to an external site. 
• Derivation of storage capacity, C~0.138N (from Hertz, Krogh, Palmer, Introduction to the theory of neural computation Download from Hertz, Krogh, Palmer, Introduction to the theory of neural computation
  ).