How can I tell luck from skill?
What else does this simulation with multiple trajectories help us understand about taking actions in the real world? Individual measurements of outcomes can be wrong, or misleading.
Think about what it actually means to measure outcomes in the real world: only a single path among the possibilities is actually taken, so chance events can strongly affect how we learn the relationship between actions and outcomes.
So, in many situations, data can be very misleading when trying to connect actions to outcomes. How do we know we didn’t just ‘get lucky’ at the previous point in time when the data was collected? How can we tell the difference between luck and skill?
This is where a simulation can be very helpful.
… Add stuff here about using the simulation to validate luck vs skill … Note that this depends on the model … Conclude that all of this relies on the simulation being ‘correct’