The agent does not know what state it transitions to after taking action. An agent goes form a set of possible initial states (Belief States). Instead of a path, we want a Conditional Plan (or strategy).
#Erratic Vacuum Example
- 8 states, with 2 goal states
- Suck action
- when applied to dirty square, sometimes even cleans adjacent
- when applied to clean square, sometimes drops more dirt
- Transition model returns possible values
- RESULTS(1, Suck) = {5, 7}
- Conditional plan
[Suck, if State = 5 then [Right, Suck] else []]
Monte Carlo Tree Search