What makes an action good?

Good outcomes and good actions

What does it even mean to take a ‘good’ action?

Let’s start by thinking about how our action-taking fits into a simulation of the outcomes.

The simplest way to do this is to apply changes to parameters in the iterations of the simulation. This can effectively simulate an action taken at any point in time, so long as the simulation logic supports this behaviour and exposes the relevant parameters.

For simplicity, we can think of the entire simulation as one operation that produces outcomes as the result of our actions.

This operation represents applying the relevant iteration logic to every state partition given it parameters, and outputting the next values of all partitions for that point in time.

From the probabilistic perspective, we only ever see one action-to-outcome pairing in most real world action-taking scenarios. The fuller range of possibilities that a simulation represents is much larger.

Simulation trajectories follow different paths through the space of possible outcomes from our actions in time.

Some paths may be more likely than others, depending on our actions. So the challenge of taking ‘good’ actions under an uncertain future is to pick those which make the the desireable outcomes more likely, or perhaps making undesireable outcomes less likely (if risk averse).

The difference between luck and skill

What else does this simulation with multiple trajectories help us understand about taking actions in the real world? Individual measurements of outcomes can be wrong, or misleading.

Think about what it actually means to measure outcomes in the real world: only a single path among the possibilities is actually taken, so chance events can strongly affect how we learn the relationship between actions and outcomes.

So, in many situations, data can be very misleading when trying to connect actions to outcomes. How do we know we didn’t just ‘get lucky’ at the previous point in time when the data was collected? How can we tell the difference between luck and skill?

This is where a simulation can be very helpful.

… Add stuff here about using the simulation to validate luck vs skill … Note that this depends on the model … Conclude that all of this relies on the simulation being ‘correct’