Module 2

Topics in deep learning

Course staff

Pontus Giselsson (Responsible), pontusg@control.lth.se 

Manu Upadhyaya (TA), manu.upadhyaya@control.lth.se

Max Nilsson (TA), max.nilsson@control.lth.se

Location and Schedule

Location: Gasquesalen, Kårhuset, LTH.

Plan to arrive between 12:00 and 12:30 on April 24th at Gasquesalen. You are welcome to store your luggage there before we head over for lunch at Restaurant Inspira (https://restauranginspira.se/ Links to an external site.).

Monday 24 April

  • 12:00-12:30 Luggage drop-of
  • 12:00-13:30 Lunch
  • 13:30-14:00 Introduction
  • 14:00-15:30 Project work
  • 15:30-15:45 Fika
  • 15:45-16:00 Example presentation
  • 16:00-17:30 Project work

Tuesday 25 April

  • 09:00-10:00 Project work
  • 10:00-10:15 Fika
  • 10:15-11:00 Presentation: Large Language Models: History and Applications. Mattias Fält, CEO Link Labs AB
  • 11:00-12:30 Project work
  • 12:30-13:30 Lunch
  • 13:30-15:00 Presentations
  • 15:00-15:30 Fika and wrap up

Aim and scope

The objective of this module is to explore a range of diverse and emerging deep learning topics in greater depth. To achieve this, we will employ an active learning teaching style through group projects, comprising 3 to 4 students. Each group will focus on one deep learning topic, selected from a predefined list of topics.

Project overview

All students must join one of the pre-generated project groups in Canvas (first come first served). There will be two groups for each topic (the list of topics can be found at the bottom of this Canvas page). NOTE: each group number is associated with a specific project, see the list below for details.

The course meeting days will primarily be devoted to preparing for group work by immersing yourselves in the chosen topic, devising a plan of action, and allocating tasks among group members. Each group will be expected to present their topic, along with three insightful questions to be explored in greater depth by the group members. Subsequently, each group will be required to submit a report and a pre-recoded presentation.

Following the final presentation and report submissions, there will be a peer review grading process. Each student will be tasked with grading two reports as described below. This means that each report will be reviewed by six to eight students. A final grade will be assigned to each report based on the reviews, and the top 10 reports with the highest average grade will be announced.

The important activities and deadlines are:

  • Course module meeting: April 24-25
  • Report and pre-recorded final presentation deadline: May 23
  • Peer-review deadline: May 28
  • Announcing winner: May 30

Attendance for all course activities is mandatory. In the event of being unable to attend, prior permission must be sought, and completing additional tasks may be necessary to obtain a passing grade.

For additional details on the presentations, report, and peer-review process, please refer to the relevant sections below.

Presentation in Lund

At the conclusion of the course meeting days in Lund, each group will have to showcase their topic and pose three questions the group members will investigate deeper. Moreover, each group should provide an overview of its work plan and outline the division of work among its group members. The presentation should be concise, lasting approximately five minutes, and the use of visual aids, such as slides, is highly encouraged.

A template (Beamer presentation tex file) is provided here Download here which you are free to use. 

Project report

The focus of this report should be on providing insight into why the chosen methodology works, rather than merely describing how it works and what it can accomplish. We encourage you to discuss the underlying enabling technologies and ideas and to offer intuition on the mechanics behind the methodology, rather than solely reporting on state-of-the-art results and outcomes.

In addition to addressing the three questions presented at the course meeting days in Lund, we ask that you conduct a survey of the topic, and explore the breakthroughs that have facilitated recent advancements and how they compare to prior state-of-the-art techniques.

The intended audience for this report is a first-year PhD student in a STEM field who is interested in method development and gaining a deeper understanding of the methodology, rather than focusing on practical implementation. While the student is assumed to have a solid foundation in deep learning (equivalent to Module 1 and beyond), they may not possess specialized knowledge of the selected topic.

The following is a suggested outline for the report:

  • Abstract: Provide a brief summary of the project's key results and findings, highlighting its significance. This should be limited to 300 words or less.
  • Introduction: Discuss the project's focus and significance in broad terms.
  • Related work: Discuss the published literature relevant to your project.
  • Questions: State your questions from the meeting presentation and your discussions and findings for each of these questions. The findings can, e.g., be in the form of a literature review, an implementation, a mathematical derivation, or a combination of the above. It may be helpful to include figures, diagrams, or tables to support your conclusions. 
  • Challenges and conclusions:
    • What have been the main challenges the community (or you as a group) faced to make this work?
    • Summarize the key findings and takeaways, as well as any relevant considerations to bear in mind when using the approach.
  • Next step: Propose possibilities for future extensions of the topic, focusing on methodology development while also considering new applications. Speculation is acceptable.
  • Authors' contributions: Detail each group member's contributions to the project. Inadequate participation may necessitate the completion of additional assignments to obtain a passing grade.
  • Pre-recorded final presentation:
    • Each group should pre-record a video presentation. The presentation should be approximately 20 minutes. While it is not mandatory for all members of the group to participate in the presentation, the emphasis should be on delivering a high-quality presentation. This may necessitate that only one or two members present on behalf of the group.
    • Upload the presentation as a private video on YouTube (or similar) and include the link in the report.

Report format

Use the NeurIPS submission style (available at https://neurips.cc/Conferences/2022/PaperInformation/StyleFiles Links to an external site.) for your report.

Submission

Submit your report here. The deadline is May 23 at 23:59.

Peer-review grading

Each student will grade (via Canvas) two other groups' reports and presentations. Below is a brief template to guide you through the peer-review process. It is meant to make recommendations for assigning between 0 and 50 points as a grade. Choose an integer in LaTeX: \lbrace0,1,2,\ldots,10\rbrace{0,1,2,…,10} in each category and take the sum:

  • Structure of the paper:
    • 0 points: Extremely poorly organized and incomprehensible.
    • 5 points:  Reasonably organized and could be improved (consider making suggestions for improvement) for easier comprehension.
    • 10 points: Extremely well organized and easily comprehensible.
  • Language:
    • 0 points: Many grammatical mistakes and typos.
    • 5 points:  Very few grammatical mistakes and typos.
    • 10 points: Very good language.
  • Originality or difficulty of the project:
    • 0 points: The project is a report of others' work and no in-depth analysis or insights have been provided.
    • 5 points:  Some own ideas have been put into the project.
    • 10 points: The project has made significant contributions to the chosen topic.
  • Theoretical depth:
    • 0 points: The report contains no theoretical considerations.
    • 5 points:  The report contains some theoretical considerations and justifications.
    • 10 points: The project contributes to the theory of the chosen topic.
  • Oral presentation:
    • 0 points: Poor presentation.
    • 5 points: A quite good presentation. 
    • 10 points: Excellent presentation.

In addition, please provide general comments about the work and your main takeaways after reading it. This could include feedback on the clarity of the writing, the relevance of the findings to the research questions, or any areas where further research or clarification may be needed.

The deadline for the peer review is May 26 at 23:59.

Topics

Each group will focus on one of the topics provided below. Each topic is followed by a list of papers and links that can serve as starting point for the project. These lists are not exhaustive and an additional literature review will be needed. To ensure exciting and diverse presentation sessions, we will allow a maximum of two groups per topic.