About this Course

8,082 recent views

This course is an introduction to sequential decision making and reinforcement learning. We start with a discussion of utility theory to learn how preferences can be represented and modeled for decision making. We first model simple decision problems as multi-armed bandit problems in and discuss several approaches to evaluate feedback. We will then model decision problems as finite Markov decision processes (MDPs), and discuss their solutions via dynamic programming algorithms. We touch on the notion of partial observability in real problems, modeled by POMDPs and then solved by online planning methods. Finally, we introduce the reinforcement learning problem and discuss two paradigms: Monte Carlo methods and temporal difference learning. We conclude the course by noting how the two paradigms lie on a spectrum of n-step temporal difference methods. An emphasis on algorithms and examples will be a key part of this course.

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Intermediate Level

Introductory computer science and data structures. Familiarity with the Python. Familiarity with basic probability and optimization.

Approx. 46 hours to complete

English

What you will learn

Map between qualitative preferences and appropriate quantitative utilities.
Model non-associative and associative sequential decision problems with multi-armed bandit problems and Markov decision processes respectively
Implement dynamic programming algorithms to find optimal policies
Implement basic reinforcement learning algorithms using Monte Carlo and temporal difference methods

Skills you will gain

Deep Learning
Markov Decision Process
Machine Learning
Reinforcement Learning
Monte Carlo Method

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Intermediate Level

Introductory computer science and data structures. Familiarity with the Python. Familiarity with basic probability and optimization.

Approx. 46 hours to complete

English

Instructor

Tony Dear

Lecturer

Computer Science

229 Learners

1 Course

Syllabus - What you will learn from this course

Week1

Week 1

6 hours to complete

Decision Making and Utility Theory

6 hours to complete

6 videos (Total 40 min), 6 readings, 2 quizzes

6 videos

Introduction to Decision Making and Reinforcement Learning1mCourse Logistics3m1.1 Rational Agents and Utility Theory9m1.2 Preferences and Axioms of Utility Theory9m1.3 Uncertain and Multi-Attribute Utilities9m1.4 Value of Perfect Information6m

6 readings

Course Syllabus10mAbout the Instructor10mAcademic Honesty Policy10mDiscussion Forum Etiquette10mPre-Course Survey 10mWeek 1 Lesson Materials10m

1 practice exercise

Utility Theory30m

Week2

Week 2

5 hours to complete

Bandit Problems

Welcome to week 2! This week, we will learn about multi-armed bandit problems, a type of optimization problem in which the algorithm balances exploration and exploitation to maximize rewards. Topics include action values and sample averaging estimation, 𝜀-greedy action selection, and the upper confidence bound. You could post in the discussion forum if you need assistance on the quiz and assignment.

5 hours to complete

3 videos (Total 36 min), 1 reading, 2 quizzes

Week3

Week 3

5 hours to complete

Markov Decision Processes

Welcome to week 3! This week, we will focus on the basics of the Markov decision process, including rewards, utilities, discounting, policies, value functions, and Bellman equations. You will model sequential decision problems, understand the impact of rewards and discount factors on outcomes, define policies and value functions, and write Bellman equations for optimal solutions. You could post in the discussion forum if you need assistance on the quiz and assignment.

5 hours to complete

6 videos (Total 36 min), 1 reading, 2 quizzes

Week4

Week 4

8 hours to complete

Dynamic Programming

8 hours to complete

6 videos (Total 42 min), 1 reading, 3 quizzes

Week5

Week 5

5 hours to complete

Partially Observable Markov Decision Processes

5 hours to complete

5 videos (Total 36 min), 1 reading, 2 quizzes

Week6

Week 6

5 hours to complete

Monte Carlo Methods

Welcome to week 6! This week, we will introduce Monte Carlo methods, and cover topics related to state value estimation using sample averaging and Monte Carlo prediction, state-action values and epsilon-greedy policies, and importance sampling for off-policy vs on-policy Monte Carlo control. You will learn to estimate state values, state-action values, use importance sampling, and implement off-policy Monte Carlo control for optimal policy learning. You could post in the discussion forum if you need assistance on the quiz and assignment.

5 hours to complete

6 videos (Total 42 min), 1 reading, 2 quizzes

Week7

Week 7

9 hours to complete

Temporal-Difference Learning

Welcome to week 7! This week, we will cover topics related to temporal difference learning for prediction, TD batch methods, SARSA for on-policy control, and Q-learning for off-policy control. You will learn to implement TD prediction, TD batch and offline methods, SARSA and Q-learning, and compare on-policy vs off-policy TD learning. You will then apply your knowledge in solving a Tic-tac-toe programming assignment.You could post in the discussion forum if you need assistance on the quiz and assignment.

9 hours to complete

5 videos (Total 36 min), 1 reading, 4 quizzes

Week8

Week 8

5 hours to complete

Reinforcement Learning - Generalization

Welcome to week 8! This module covers n-step temporal difference prediction, n-step SARSA (on-policy and off-policy), model-based RL with Dyna-Q, and function approximation. You will be prepared to implement n-step TD learning, n-step SARSA, Dyna-Q for model-based learning, and use function approximation for reinforcement learning. You will apply your knowledge in the Frozen Lake programming environment. You could post in the discussion forum if you need assistance on the quiz and assignment.

5 hours to complete

4 videos (Total 39 min), 2 readings, 2 quizzes

Frequently Asked Questions

When will I have access to the lectures and assignments?
Access to lectures and assignments depends on your type of enrollment. If you take a course in audit mode, you will be able to see most course materials for free. To access graded assignments and to earn a Certificate, you will need to purchase the Certificate experience, during or after your audit. If you don't see the audit option:
The course may not offer an audit option. You can try a Free Trial instead, or apply for Financial Aid.
The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
What will I get if I purchase the Certificate?
When you purchase a Certificate you get access to all course materials, including graded assignments. Upon completing the course, your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
Is financial aid available?
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.

More questions? Visit the Learner Help Center.

Decision Making and Reinforcement Learning