How does an action enter a simulation?
What does it even mean to take a ‘good’ action?
Let’s start by thinking about how our action-taking fits into a simulation of the outcomes.
The simplest way to do this is to apply changes to parameters in the iterations of the simulation. This can effectively simulate an action taken at any point in time, so long as the simulation logic supports this behaviour and exposes the relevant parameters.
For simplicity, we can think of the entire simulation as one operation that produces outcomes as the result of our actions.
This operation represents applying the relevant iteration logic to every state partition given it parameters, and outputting the next values of all partitions for that point in time.
References and further reading
Hardwick, Robert J. 2026a. “How are a partition’s next values calculated?” umbralcalculations, November. https://umbralcalc.github.io/posts/how_are_a_partitions_next_values_calculated.html.
Hardwick, Robert J. 2026b. “How can I trust the actions I take in simulations?” umbralcalculations, August. https://umbralcalc.github.io/posts/how_can_i_trust_the_actions_i_take_in_simulations.html.
Hardwick, Robert J. 2026c. “What is a state partition?” umbralcalculations, November. https://umbralcalc.github.io/posts/what_is_a_state_partition.html.
Sutton, Richard S., and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. 2nd ed. MIT Press.
Questions whose answers cite this one