You submit your solution : here

In this elective task you will build a complete — but simplified — autonomy pipeline for a mobile robot: perception, state estimation, planning, and control, integrated in a small 2D simulation. Unlike elective tasks A–C, which each focuses on one part of the course, this task goes broad and touches all of the course modules.

Building all of this from scratch would not be feasible within the scope of one elective task. Therefore, this task requires the use of a large language model (LLM) as your design and coding partner (e.g., ChatGPT, Claude, Gemini, or GitHub Copilot). Part of the learning objective is to practice working with an LLM on a technical system, and — importantly — to use your knowledge from the course to critically evaluate, verify, and correct what the LLM produces.

NOTE: The point of this task is not to produce a polished software product, and not to show that an LLM can write code. The point is to demonstrate that you understand the pipeline well enough to specify it, spot the LLM’s mistakes, and analyze the behavior of the resulting system using the theory from the course. The report is the primary assessed artifact, not the amount of code.

Scope and time budget

This task is deliberately broader than elective tasks A–C: it spans all of the course modules and is intended for students who feel confident with the material. If the mandatory assignments went quickly for you, the time saved there can be well invested here. Like the rest of the course it is graded pass/fail. Some guidance to keep the effort well spent:

Aim for the simplest implementation that lets you investigate the questions in the task. Do not spend time improving software architecture, visualization, planner performance, or robustness unless it contributes directly to your analysis. One Python script, or a handful of modules, is perfectly acceptable.
A guaranteed-sufficient baseline for a pass is: grid-based A*, pure pursuit tracking, nearest-neighbor data association, a basic EKF, one simple obstacle world, and one stress test. You are welcome — encouraged, even — to choose more ambitious alternatives (RRT, LQR, sophisticated gating, continuous replanning, multiple stress tests) where they interest you, but know that the baseline is enough: ambition should go where you find it interesting, and the analysis is what matters.
Escape clause: if you find yourself spending far more time than intended without a complete pipeline, fall back to the baseline, get a minimal integrated system running, and prioritize the verification and analysis work.

Prerequisites

Python 3 with numpy and matplotlib (no ROS, Matlab, or GPU required).
Access to an LLM. Any capable model/interface is fine. Keep in mind that you must be able to save and submit your conversation transcripts.

The system you will build

The robot is a unicycle (differential-drive) model:

$LaTeX: \begin{align} \dot{x} &= v\cos\theta,\\ \dot{y} &= v\sin\theta\\ \dot{\theta} &= \omega \end{align}$

with control inputs LaTeX: v (speed) and $LaTeX: \omega$ (turn rate), operating in a 2D world containing a set of circular obstacles and a set of point landmarks at known positions.

You will implement (with LLM support) the following pipeline:

sensors
(simulated)

$LaTeX: \longrightarrow$

Perception
(association)

$LaTeX: \longrightarrow$

Estimation (EKF)
(localization)

$LaTeX: \longrightarrow$

Planning
(A* / RRT)

$LaTeX: \longrightarrow$

Control
(tracking)

$LaTeX: \longrightarrow$

robot

D.0 Simulator

A simple discrete-time simulator that:

integrates the unicycle dynamics (e.g., Euler at 50 Hz);
simulates noisy odometry (velocity inputs corrupted by Gaussian noise);
simulates a range–bearing sensor: for each landmark within a maximum range and field of view, it returns range and bearing corrupted by Gaussian noise, plus occasional outliers and dropouts;
provides a plot/animation of ground truth, estimate, plan, and obstacles.

Let the LLM generate most of this — but you choose the noise levels, rates, and geometry, and you must be able to justify them.

D.1 Perception / measurement processing

From the raw range–bearing returns, associate measurements to landmarks (data association) and reject outliers (e.g., a validation gate). Nearest-neighbor association is the expected level — but document its failure modes. (You are not expected to build a realistic perception front-end such as camera or LiDAR processing.)

D.2 State estimation

An Extended Kalman Filter that fuses odometry (prediction step) with the landmark range–bearing measurements (update step) to estimate \((x, y, \theta)\) and its covariance. This is the same machinery as in the GNSS-aided INS assignment, in a lower-dimensional setting.

Verify the filter yourself: check that the estimation error is consistent with the reported covariance (e.g., plot errors with \(\pm 3\sigma\) bounds, or compute NEES). LLMs frequently get Jacobians, angle wrapping, or noise handling subtly wrong — this is where you catch it.

D.3 Planning

A global planner from start to goal among the obstacles: A* on a grid or an RRT variant — your choice (A* is the baseline). The plan must respect an inflation margin around obstacles (why is this needed when the plan is executed using an estimated pose?).

D.4 Control

A path tracking controller executing the plan on the unicycle: pure pursuit (baseline) or a linearization-based design such as LQR on the error dynamics, connecting to elective task C — your choice. The controller acts on the EKF estimate, not on ground truth.

D.5 Closing the loop and stress testing

Run the full loop: sense → estimate → (re)plan → control. Then inject one of the following failures and analyze what happens (doing more is welcome but not required):

A landmark/measurement dropout window (compare with the GPS-outage study in Assignment 2 / Elective B).
Model mismatch: simulate with different wheel radius / actuator gain than the filter and controller assume.
A biased sensor or wrongly specified noise covariance.
A moving obstacle that the planner does not know about.

For the injected failure: which module degrades first, how does the failure propagate through the pipeline, and which course concepts explain the observed behavior?

Working with the LLM — requirements

This is the core of the task. You must:

Document the LLM(s) you used. State in the report: which model(s) and version (e.g., “GPT-5, ChatGPT web interface, October 2025”), which interface/tool (chat, IDE assistant, agentic CLI), any relevant settings, and roughly how the work was divided between you and the LLM. If you switch models during the task, note where and why.
Keep a prompt log. Save the full transcript(s) of your LLM interactions and submit them as an appendix (export, PDF, or share links). Annotate only the 5–10 most important exchanges (what you asked, why, and what you did with the answer); the rest of the transcript can be supplied unannotated. Do not spend time curating your chat history.
Verify the LLM’s output: at least three verification cases. Do not take the LLM’s code or explanations on trust. For at least three technically substantive pieces of LLM output, perform and document a rigorous verification that either confirms or refutes it. Examples of suitable verifications:
- check the EKF measurement Jacobian against numerical differentiation;
- test EKF consistency (error vs. \(\pm 3\sigma\) bounds, NEES);
- verify angle wrapping of the bearing innovation with an adversarial test case (e.g., a bearing crossing \(\pm\pi\));
- check that the planner’s output respects the inflation margin and that the tracking controller can actually follow it (nonholonomic feasibility);
- unit-test data association / outlier rejection on constructed edge cases.
For each case, document: what the LLM produced (quote it), how you tested it, the outcome, and the course concepts behind the test. If a verification refutes the output — LLMs classically slip on Jacobian signs, angle wrapping, noise-covariance placement, and nonholonomic constraints — also document your fix and evidence that it works. Genuine refutations count as the strongest verification cases, but a well-designed test that confirms correct output is fully acceptable: the graded skill is verification, not whether the LLM happened to err.
Stay in charge of the design. All top-level design decisions (state vector, noise models, planner choice, controller structure, tuning) are yours and must be motivated in the report in your own words.

Deliverables

Code (a small repository or zip) with a README explaining how to run it. It should run with plain Python + numpy + matplotlib.
A video (or animated GIF) of the full closed-loop system: ground truth vs. EKF estimate with covariance ellipse, the planned path, and the failure scenario from D.5.
A report (guideline: 4–8 pages plus appendix) containing:
- which LLM(s) you used and how (see above);
- a brief system description with your design choices and motivations;
- EKF consistency evaluation (error vs. covariance);
- the stress-test analysis from D.5;
- the three (or more) verification cases, including any refutations and fixes;
- answers to the questions below;
- the annotated prompt log as an appendix.

Questions

Discuss these in the report:

Where in the pipeline was the LLM most useful, and where was it most dangerous? What made the difference?
Which parts could you not have verified without the theory from this course? Give concrete examples.
Your EKF consistency results: is the filter over- or under-confident, and what tuning choices drive this?
In the stress test, how did errors propagate between modules? Could a different architecture (e.g., replanning frequency, integrated estimation–control design) mitigate this?
(Optional.) The pipeline here is modular and classical. Based on your experience in this task, comment on the trade-off between such modular stacks and end-to-end learned approaches for safety-critical autonomous systems.
Reflect: if you did this task again, how would you change the way you use the LLM (prompting strategy, verification workflow, division of labor)?

Grading

The task is graded pass/fail, like the rest of the course. Assessment is based on the report: depth of analysis, correctness of the theory used to evaluate the system, and quality of the verification cases — not on code volume or visual polish. The baseline route described under Scope and time budget is fully sufficient; more ambitious alternatives are welcome but do not affect the outcome. A partially working system with an insightful failure analysis is worth more than a flawless system with a shallow report. Submitting LLM-generated report text without your own analysis is not acceptable; the prompt log makes your contribution traceable.

Alternative to D: as with elective C, if you want to adapt the scope (e.g., a 3D or ROS-based variant, or a different robot model), contact us and we will find something together.

Hint: You find some advice on how to get started in this presentation