Deep Learning

Textbook material

Given the interest in deep learning it is not surprising that there are a number of textbooks (physical or e-versions) on the subject. The bible is Goodfellow, Bengio, and Courville’s Deep Learning Links to an external site. book which is available online. However, it is a brick and quite theoretical. It fits well for those of you that want the details and to be able to go deep. For the assignment in this course, the book Deep learning with Python Links to an external site. by François Chollet fits very well as it takes you from the basics. A list of books is available here Links to an external site..

Introduction to deep learning

If you are completely new to neural networks, take a look the first chapter of Neural Networks and Deep Learning Links to an external site. by Michael Nielsen. Here you will be introduced the neuron model, how they can be combined into networks, how to define a basic cost function and how to learn the parameters using gradient descent.

Chapter 3 in the same book Links to an external site. discusses the cross-entropy cost function, regularisation methods, ways to initialise the weights in the network and ways to choose good hyper parameter. All seems like familiar concepts to you? Skip over this, but if not take a look. The important point here is to get familiar with the concepts and ideas not to know every detail. Deep learning is very much an engineering approach that takes lots of experience to do well, but having a fundamental theoretical understanding helps this process.

An alternative book is Deep learning with Python Links to an external site. by François Chollet which gives a slightly more applied approach. So, if you are more of a learning by doing type, we recommend the first three chapters which are available for download here. You find the corresponding set of Jupyter notebooks here https://github.com/fchollet/deep-learning-with-python-notebooks Links to an external site.. 

To start with, select the notebook 2.1-a-first-look-at-a-neural-network.ipynb and step through it and map what you find there to the general concepts you learned about above. NOTE: The training will take quite some time so let it simmer in the background and do not be afraid to interrupt it when you have got what you want from it. You can see everything that happens and why in Chapter two in Deep learning with Python Links to an external site. (available online).

 

Natural Language Processing

Deep Learning for Natural Language Processing (Richard Socher, Salesforce) Links to an external site.Deep Learning for Natural Language Processing (Richard Socher, Salesforce)

https://en.wikipedia.org/wiki/Word2vec Links to an external site.

http://cs224d.stanford.edu/syllabus.html Links to an external site.

Slides from talk: DL for NLP, Socher Links to an external site.

Slides from Sochers ML summer school Links to an external site.

https://www.linkedin.com/pulse/best-ai-algorithms-sentiment-analysis-muktabh-mayank

Deep Learning for Natural Language Processing: Tutorials with Jupyter Notebooks Links to an external site., by John Kron. Signup required, 10 day free trial.

ELMo, said to be state-of-the-art for word embeddings Links to an external site.

Additional material Links to an external site.