Module 1

Teachers

Lennart Svensson lennart.svensson [at] chalmers.se (examiner)
Georg Hess georghe [at] chalmers.se (teaching assistant)

Prerequisites

Anyone admitted to WASP-AI should be able to attend this module (and is required to). However, students with a strong background in Python, machine learning, neural networks and convolutional neural networks are likely to find the module easy, whereas other students will have a harder time, but hopefully also find that it contains some opportunity for reflection.


Aim

In this first course module, we aim to harmonize the class and ensure that all students understand the basic concepts and tools in deep learning.

Schedule

The module contains three parts:

  1. Preparations, before 28/3.
  2. Two days of classes, 28-29/3.
  3. Home assignments, after 29/3. The deadline is 21/4. 

Additional details follow. 

Preparations 

Before coming to class, students are expected to make the following preparations:

  • Get a Canvas account; see the front of this Canvas page. 
  • Fill out this form Links to an external site. as soon as possible.
  • Watch lectures 1, 2, 3, 4, 5 and 6. Note that we have created one module for each lecture (see Module 1, L1 to Module 1, L6) and that you can find all lectures under Modules. It is not mandatory to watch the lectures, but you are expected to be familiar with the material covered in the videos and watching the videos may be the easiest way to ensure this. Even though we recommend that you watch the videos, we have also prepared reading directions as a useful complement to the videos.
  • Complete quizzes 1, 2 and 3. Note that completing the quizzes are mandatory, and you can find them under Module 1 (or using the direct links: Quiz 1, Quiz 2 and Quiz 3).
  • The quizzes closely follow the video lectures and check that you are familiar with the course material.
  • Go through the Python preparations, including the Python crash course (which is available as yet another module under Modules). It is not mandatory to go through the Python preparations, but we strongly recommend you to do so unless you have a solid background in Python programming.  
  • Join a project group on Canvas. There will be 2 students in each group and we strongly urge you to team up so that there is a maximum of 1 student with GPU access. Unless you know who you want to work with, you can use Forming groups for home assignments to reach out to your peers. Once you have identified a partner, you should join the same group under People.

The teaching style of the module is called 'Flipping the classroom' and requires a significant amount of preparation from the students, as described below in the 'Organisation' section.

Note: your preparations are of vital importance in order for the sessions to be useful.

2-days of classes
You are expected to attend our classes at Chalmers University of Technology:

  • 28/3 13:00 - 15:00
  • 28/3 15:15 - 17:15
  • 29/3 09:00 - 11:30
  • 29/3 13:00 - 15:30

The sessions are performed in SB Multisal, https://maps.chalmers.se/#97dc2ff0-bd34-425f-bfd9-10dbc49b1946 Links to an external site.. 

We will also offer some social activities (essentially free food) that you are welcome to attend: 

 

Home assignments 

A mandatory home assignment is part of the examination. The assignment is posted shortly after the physical session. 

 

 

Course administration and teaching style

Canvas

All course administration is done through this webpage. The general info is given on this very page, and the more specific details are found in the 'pages' section found in the column to the left. If you have any questions, please email Georg Hess, georghe [At] chalmers.se, or if the question is more general, post a thread in the Canvas discussion forum.

Flipping the classroom

The course is given in a flipping-the-classroom style, and it may therefore be useful for you to understand what that is and why we want to use it. By doing so, we also motivate why it is important that you show up well-prepared to class. 

In short, the idea behind flipping the classroom is that you should get a basic understanding of the material before the class, such that the time with the teacher is instead focused on deepening your understanding and learning how to analyse and apply what you have learnt. There are several arguments in favour of this approach, for instance, students show up better prepared for the in-class discussions and have the support of their peers and a teacher when they perform higher-level tasks (analyse, apply, etc). The main argument, however, is that studies show that it improves learning.

An excellent overview of flipping the classroom can be found here: http://cft.vanderbilt.edu/guides-sub-pages/flipping-the-classroom/ Links to an external site.. They list the key elements of flipping the classroom (here mixed with comments regarding how this is done in this course):

 

  1. Provide an opportunity for students to gain first exposure prior to class.
    In this module, you are expected to watch the online videos on Scalable learning Links to an external site. before class.

  2. Provide an incentive for students to prepare for class.
    In this case, the main incentive is that this is required in order for you to make the most of this learning opportunity. 

  3. Provide a mechanism to assess student understanding.
    This is useful for students but also for the teacher since it provides material that enables the teacher to tailor in-class activities to what the students need. For this, we partly rely on the results from the video quizzes, but hopefully, students will also make use of the possibility to ask questions using the buttons at the bottom of the video player in the online system that we use.

  4. Provide in-class activities that focus on higher-level cognitive learning
    In class, we devote our time to active learning, where students can achieve deeper understanding. There are many different types of activities we can devote our time to, and we will try to design activities that fit the students and the topic as well as possible. 

For those interested, we've put together two videos on flipped classroom teaching at Masters courses at Chalmers:
An introduction to flipped classroom teaching Links to an external site.An introduction to flipped classroom teachingFlipping a Masters course Links to an external site.Flipping a Masters course

Syllabus

The modules consist of three sections, where the idea is that the material in Sections 1 and 2 is covered during the first day (and in the corresponding material to read up on at home). After that, we cover Section 3 on the second day. The content of the three sections is listed below. 

  1. Fundamental knowledge of machine learning, Python and the components of a neural network.
    • A crash course in Python (Optional)
    • Overview of different types of machine learning: SL vs UL vs RL.
    • Basic principles behind supervised learning, including empirical risk and cross-entropy.
    • Basics of feedforward networks, common activation functions and when to use them
  2. Basics of how to train a neural network
    • Computational graphs, backpropagation and Stochastic gradient descent
    • Train/validation/test sets
    • A basic recipe for machine learning
    • Regularisation
  3. Convolutional neural networks
    • An introduction to the basic components (convolutions, strides, pooling layers)
    • An overview of common architectures
    • Relation to feed-forward networks (shared weights, number of parameters)
    • Transfer learning
    • Adversarial examples and the idea of maximising variables with respect to input data
    • CNN's for different applications such as
      • Semantic segmentation: shared calculations and transposed convolutions
      • Object detection: single shot detectors, their loss functions and basic principles
      • Other data types such as text, music, LIDAR point clouds, etc

 


Literature

The required theory is given in the video lectures, but for further exploration, we recommend:

Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016, which is available online Links to an external site.. See Reading directions and slides from videos for additional details.