Reinforcement Learning Assignment 1 Solution

Description

• Introduction

The goal of this assignment is to do experiment with Dynamic Programming(DP), including iterative policy evaluation, policy iteration and value iteration. Your goal is to implement DP methods and test them in the small gridworld men-tioned in the slides of Lecture 3.

• Small Gridworld

Figure 1: Gridworld

As shown in Fig.1, each grid in the Gridworld represents a certain state. Let s_t denotes the state at grid t. Hence the state space can be denoted as S = fs_tjt 2 0; ::; 35g. S₁ and S₃₅ are terminal states, where the others are non-terminal states and can move one grid to north, east, south and west. Hence the action space is A = fn; e; s; w g. Note that actions leading out of the Gridworld leave state unchanged. Each movement get a reward of -1 until the terminal state is reached.

A good policy should be able to nd the shortest way to the terminal state randomly given an initial non-terminal state.

• Experiment Requirments

Programming language: python3

You should build the Gridworld environment and implement iterative pol-icy evaluation methods and policy iteration methods. Then run the two methods to evaluate and improve an uniform random policy (nj ) = (ej ) = (sj ) = (wj ) = 0:25

• Report and Submission

Your report and source code should be compressed and named after \stu-dentID+name”.

The les should be submitted on Canvas before Apr. 3, 2020.

3