Recommended exercises to do on your own:

Markov Chains in Discrete Time: 1.5, 1.6, 1.8.
Markov Chains in Continuous Time: 1.12, 1.15.
Queuing Theory: 2.2, 2.5, 2.6, 2.14.
Inventory Theory: 3.1, 3.2, 3.6, 3.8, 3.9
Marginal Allocation 4.1, 4.3, 4.4
Deterministic Dynamic Programming: 5.1, 5.2, 5.3
Stochastic Dynamic Programming: 5.4 (used in one of the exercise classes), 5.6, 5.8, 5.10-12
Markov Decision Processes: 6.2, 6.3, 6.4, 6.9 a,c

Exercises that will be solved in class

Exercise class 1: Markov processes 1.4a,b, 1.11, Component manufacturers (problem_ovn1_manufacture), Cat and Mouse problem (problem_ovn1_catnmouse).
Exercise class 2: Queueing theory 2.10a-d, 2.5a-c, and 2.11 a-b.
Exercise class 3: Jackson networks 2.1, Exam 20120612-1
Exercise class 4: Marginal allocation - The satellite. The Satellite. (Manuscript.)
Exercise class 5: Inventory theory, Taxi-driver EOQ with discount (problem_ovn5), 3.3 Newsvendor
Exercise class 6: Inventory theory 3.5, deterministic review model - Boat problem (problem_ovn6_boat), selling oil Exam 20080530-2 (problem_ovn6_oil)
Exercise class 7: Dynamic programming, Knapsack problem 5.5, Distributing Scientists from IOR
Exercise class 8: DynP 4.4 extended version, Markov decision problems Exam 20120612-3 if time allows
Exercise class 9: Markov decision problems with discount, Taxi Driver Exam 20080530-5 (problem_ovn9_taxi), Exam 20160111-4b