Module 1

Teachers

Lennart Svensson lennart.svensson [at] chalmers.se (examiner)
Jakob Lindqvist jakob.lindqvist [a] chalmers.se (teaching assistant)


Aim

In this first course module, we aim to harmonize the class and ensure that all students understand the basic concepts and tools in deep learning.

Schedule

The module contains three parts:

  1. Preparations, performed before 13/4.
  2. Two days of classes, 13-14/4.
  3. Home assignments performed after 14/4.

Additional details follow. 

Preparations 

Before 13/4, the students are expected to make the following preparations:

  • Get a Canvas account, see the course home page.
  • Fill out this form Links to an external site. as soon as possible.
  • Watch lectures 1, 2, 3, 4, 5 and 6. Note that we have created one module for each lecture (see Module 1, L1 to Module 1, L6) and that you can find all lectures under Modules. 
  • Complete quizzes 1, 2 and 3. Note that completing the quizzes are mandatory and you can find them under Module 1. 
  • Go through the Python preparations, including the Python crash course (which is available as yet another module under Modules).
  • Join a project group on Canvas. There will be 2 students in each group and we strongly urge you to team up so that there is a maximum of 1 student with GPU access. Unless you know who you want to work with, you can use Forming groups for home assignments to reach out to your peers. Once you have identified a partner, you should join the same group under People.

The teaching style of the module is called 'Flipping the classroom' and it requires a significant amount of preparations from the students, as described below in the 'Organisation' section.

Note: your preparations are of vital importance in order for the sessions to be useful.

2-days of online classes
Mandatory sessions at Chalmers University of Technology:

  • 13/4 09:00 - 11:30
  • 13/4 13:00 - 15:30
  • 14/4 09:00 - 11:30
  • 14/4 13.00 - 15.30

The sessions are performed online, see Calendar for links. 

Home assignments 

A mandatory home assignment is part of the examination.

Prerequisites

Anyone who has been admitted to WASP-AI should be able to attend this module (and are in fact required to). However, students with a strong background in Python, machine learning, neural networks and convolutional neural networks are likely to find the module easy, whereas other students will have a harder time, but hopefully also find that it contains some opportunity for reflection.

 

Organisation

The module comprises:

  • A Python crash course  with instruction videos and related Python preparations to provide the Python skills needed for the course. 
  • Video lecture 1, 2, 3, 4, 5 and 6 posted as six separate modules (see Module 1, L1 to Module 1, L6)
  • It is not mandatory to watch the lectures, but you are expected to be familiar with the material covered in the videos and watching the videos may be the easiest way to ensure this. Even though we recommend that you watch the videos, we have also prepared reading directions as a useful complement to the video.
  • Quizzes 1, 2 and 3. The quizzes closely follow the video lectures and check that you are familiar with the course material, they can be found under Module 1.
  • Two days of classes (see calendar) that contain a combination of:
    • Short lectures with the objective to refresh the material taught in the videos.
    • Practice sessions where students work on the material in groups of 4 students.
  • Two quizzes
  • One mandatory group assignment on CNN's: Home assignment - HA1

Canvas

All course administration is done through this webpage. The general info is given in this very page and the more specific details are found in the 'pages' section found in the column to the left. If you have any questions, please email Jakob Lindqvist at jakob.lindqvist [At] chalmers.se or if the question is more general, post a thread in the Canvas discussion forum.

Flipping the classroom

The course is given in a flipping-the-classroom style and it may therefore be useful for you to understand what that is and why we want to use it. By doing so we also motivate why it is important that you show up well prepared to class. 

In short, the idea behind flipping the classroom is that you should get a basic understanding of the material already before the class, such that the time with the teacher is instead focused on deepening your understanding and learning how to analyse and apply what you have learnt. There are several arguments in favour of this approach, for instance that students show up better prepared for the in-class discussions and that they have the support of their peers and a teacher when they perform higher-level tasks (analyse, apply, etc). The main argument, however, is that studies show that it improves learning.

An excellent overview on flipping the classroom can be found here: http://cft.vanderbilt.edu/guides-sub-pages/flipping-the-classroom/ Links to an external site.. They list the key elements of flipping the classroom (here mixed with comments regarding how this is done in this course):

 

  1. Provide an opportunity for students to gain first exposure prior to class.
    In this module, you are expected to watch the online videos on Scalable learning Links to an external site. before class.

  2. Provide an incentive for students to prepare for class.
    In this case, the main incentive is that this is required in order for you to make the most of this learning opportunity. 

  3. Provide a mechanism to access student understanding.
    This is useful for students but also for the teacher since it provides material that enables the teacher to tailor in-class activities to what the students need. For this we partly rely on the results from the video quizzes, but hopefully students will also make use of the possibility to ask questions using the buttons at the bottom of the video player in the online system that we use.

  4. Provide in-class activities that focus on higher level cognitive learning
    In class we devote our time to active learning, where students can achieve deeper understanding. There are many different types of activities that we can devote our time to, and we will try to design activities that fit the students and the topic as good as possible. 

For those interested, we've put together two videos on flipped classroom teaching at Masters courses at Chalmers:
An introduction to flipped classroom teaching Links to an external site.An introduction to flipped classroom teachingFlipping a Masters course Links to an external site.Flipping a Masters course

Syllabus

The modules consists of three sections where the idea is that the material in Section 1 and 2 are covered during the first day (and in the corresponding material to read up on at home). After that, we cover section 3 during the second day. The content of the three sections are listed below. 

  1. Fundamental knowledge about machine learning, Python and the components in a neural network.
    • A crash course in Python (Optional)
    • Overview of different types of machine learning: SL vs UL vs RL.
    • Basic principles behind supervised learning, including empirical risk and cross-entropy.
    • Basics of feedforward networks, common activation functions and when to use them
  2. Basics of how to train a neural network
    • Computational graphs, backpropagation and Stochastic gradient descent
    • Train/validation/test sets
    • Basic recipe for machine learning
    • Regularisation
  3. Convolutional neural networks
    • An introduction to the basic components (convolutions, strides, pooling layers)
    • An overview of common architectures
    • Relation to feed-forward networks (shared weights, number of parameters)
    • Transfer learning
    • Adversarial examples and the idea of maximising variables with respect to input data
    • CNN's for different applications such as
      • Semantic segmentation: shared calculations and transpose convolutions
      • Object detection: single shot detectors, their loss functions and basic principles
      • Other data types such text, music, LIDAR point clouds, etc

 


Literature

Required theory is given in the video lectures but for further exploration we recommend:

Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016, which is available online Links to an external site.. See Reading directions and slides from videos for additional details.