¶¶Òõ´«Ã½ÔÚÏß

Duration: 3Ìýhours, 59 minutes

This module introduces the modeling and optimization foundations for sequential decision-making in their simplest form: deterministic decision processes with discounted rewards. We begin with states, actions, transitions, and rewards as a language for representing decision problems over time. We then develop value functions and discounted optimality equations as tools for optimizing long-term return. The goal is to build intuition for why dynamic programming is correct in the simpler setting of deterministic decision processes before introducing stochastic transitions, learning from sampled experience, and bootstrapping in later modules.

Duration: 2Ìýhours, 50 minutes

This module adds stochasticity to the deterministic picture developed in the previous module. Learners continue with the surprise-quiz example, now with uncertain outcomes: studying usually helps but may not always help, and relaxing may reduce preparation but may not always do so. The module first introduces stochastic transitions as probability distributions over next states, then studies Markov chains as stochastic systems without choices and finally adds actions to obtain Markov decision processes. The goal is to make expected discounted reward, policies, and Bellman equations feel like natural extensions of the deterministic setting.Ìý

Duration: 2 hours, 26 minutes

This module focuses on known-model optimization. Learners use Bellman equations as computational tools for policy evaluation, policy improvement, value iteration, policy iteration, and linear programming formulations of discounted MDPs.

Duration: 2Ìýhours, 21 minutes

This module begins the transition from planning to reinforcement learning. Inplanning, the MDP model is known and Bellman backups compute expectationsexactly. In reinforcement learning, the model is replaced by sampledexperience. Learners first view reinforcement learning as sample-based dynamicprogramming, then study rewards, uncertainty, agent--environment interaction,bandit estimation, exploration versus exploitation, Monte Carlo policyevaluation, and Monte Carlo control.

Duration: 1Ìýhour, 42 minutes

This module completes the tabular reinforcement-learning part of Course 1. Module 4 introduced sample-based learning through bandits and Monte Carlo methods. Module 5 introduces temporal-difference learning: updating after one sampled transition by combining an observed reward with a bootstrapped value estimate. The module ends by summarizing tabular reinforcement learning and motivating the transition to function approximation and deep RL.

Duration: 2 hours, 12 minutes

• The exam is non proctored.
• It is a two-hour exam.
• You may submit your exam only once.
• The exam contains only multiple choice questions.
• You are not allowed to use any notes or access other websites when you take your exam.