Finite MDPs
Recall from the previous concept that in a finite MDP, the state space\mathcal{S}S(or\mathcal{S}^+S+, in the case of an episodic task) and action space\mathcal{A}Amust both be finite.
Thus, while the CartPole-v0 environment does specify an MDP, it does not specify a finite MDP. In this course, we will first learn how to solve finite MDPs. Then, later in this course, you will learn how to use neural networks to solve much more complex MDPs!
Table of environments (OpenAI Gym)
https://github.com/openai/gym/wiki/Table-of-environments
