Furhat: Conversational System Project

The assignment covers the areas knowledge, interaction and collaboration. Make sure to read this page first without going into all the links as a first step.

The links below give you brief introductions to (some aspects of) the areas covered in the second part of the course. You will be diving deep into one type of interaction in the project, that between a robot and one or more humans users but interaction is a much broader area than this as you can see below. Research on collaboration is conducted in several contexts such as multi agent systems and game theory but also in the context of human-robot interaction as discussed below. You are not required to present the topics below (like you did in the first part) but they might come in handy.

 

Conversational systems

For this assignment you will work specifically with conversational systems. For an introduction please watch Prof. Gabriel Skantze's first lecture in a dedicated course on the topic Links to an external site.. 

 

Project description

In this group assignment you will design, implement and evaluate a conversational system. You will work in the new groups called "WASP_AS2_P2 X". The first person listed in the group is the contact person and is responsible for calling the first meeting and communication with course leaders when needed.

Your group will use the simulator for the social robot Furhat as a platform for the work. Below are instructions for how to install its SDK and virtual robot simulator.

Your group are expected to come up with your project idea. Look at the bottom of this page for some examples from 2018. 

Remember that the focus is on knowledge, interaction and collaboration in this module and not, for example, learning and perception. The latter two are easy to connect to a project but they should be icing on the cake in that case rather than the cake/focus.

Note: It is not enough to change one of the existing examples from selling fruits to books and just hack Furhat SDK. You need to show us that you have acquired some understanding of the design of conversational  systems. 

Remember that you do not have much time so divide the work and focus on what is important.

The project will be conducted in two parts described below, each with its own deliverables. Please consult the deliverables for Part 2 for the final deliverables.

The project will be done completely in simulation. No Furhat hardware will be available.

At your first meeting with the group, we want you to start with a short presentation about your research projects. 

 

Installations and preparations

Follow the instruction on the page Furhat: Preparations. As mentioned there, to save time for the group:

We strongly suggest that you play around with Furhat SDK before you decide on a project idea so that you have a better understanding for what is easy and what is hard. One way to do this is to work through an optional assignment, Furhat: Extending Fruit Seller, where you get some suggestions for extensions for the basic skill that you looked at in Furhat: Preparations. 

Gabriel mentions a number of tools and frameworks that might be useful at 1:27:05 in the lecture video above. Links to an external site.

If you have problems ask in the Discussion forum here on Canvas to see if someone else can help you. Do not contact Furhat directly. There are too many of you for that.

Part 1

This first part is a form of check point. We want you to submit a project proposal that we can give feedback on before you finish the design and get started with the implementation and evaluation work.

Remember that the assignment is meant to cover knowledge, interaction and collaboration with a focus on conversational systems. This means that it should be clear from the proposal that this is the focus rather than, for example, learning or perception. If (when) you have to scale down your idea in the actual implementation you should be left with the core topics rather than something else.

Submit a brief (~0.5 page) description of your project idea.

This first part in the project introduces a deadline that was not present in the original schedule but we have removed one assignment. Also this intermediate deadline for the project should not add work or change the schedule because you anyway need to decide what to do first and you need enough time to be able to implement and evaluate this on other groups.

Submission: Furhat: part 1

 

Part 2

This is where you do the actual project work.

The deliverables in the project should be collected in a zip-file containing:

  1. A slide deck presenting your system (see below for the requirements for the examination slot)
  2. A video demonstrating your system
  3. Code for the system
  4. A brief report that
    1. states what each member contributed with and acknowledges that all members that are listed as authors have contributed fairly to the work,
    2. (briefly) describes the system design,
    3. outlines how your design draws from the literature (i.e. show that you learned more than how to hack Furhat SDK),
    4. evaluation where the system has been tested on at least one other group in the course, and
    5. discusses lessons learned about conversational systems and knowledge, interaction and collaboration in general.
  5. A document discussing your experience with using the Furhat SDK and any suggestions for improvements.

Submission: Furhat: part 2

 

Examination sessions (Dec 14-18)

Each group is expected to take part in one of the examination zoom sessions during the week Dec 14-18 where you present your work with a few slides and ideally demonstrate your system or at least show your video. Each session will have 3-4 groups presenting to each other.

You have 20min to present and demonstrate your work including everything. This is followed by 10min of discussion. 

The minimum requirement on each participant when it comes to demoing is that you are able to run the Fruit Seller example on a computer you have access to.

Your group will be able to sign up for a slot to present during the week Dec 14-18 within the coming weeks.

 

Examples from 2020

Below are some examples from the WASP course in 2018.