From root: How do I automate taking actions with simulations?

What is discounted future reward?

The ‘discounted future reward’ is a quantity we can specify that a simulation trajectory will have accumulated into the future, accounting for increasing distance into the future by ‘discounting’ it gradually with a weighting.

We are using this concept of discounted future reward in the same way that it is used in reinforcement learning.

The idea is that, as you go further into the future, the importance of the reward you have accumulated by then is increasingly irrelevant to actions you might take at the present moment.

References and further reading

Hardwick, Robert J. 2026a. “How do I automate taking actions with simulations?” umbralcalculations, September. https://umbralcalc.github.io/posts/how_do_i_automate_taking_actions_with_simulations.html.

Hardwick, Robert J. 2026b. “How do I set up a simulation to learn a policy?” umbralcalculations, August. https://umbralcalc.github.io/posts/how_do_i_set_up_a_simulation_to_learn_a_policy.html.

Sutton, Richard S., and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. 2nd ed. MIT Press.

Wikipedia contributors. n.d. “Reinforcement Learning.” Wikipedia. https://en.wikipedia.org/wiki/Reinforcement_learning.