Backtesting and other challenges

Backtesting actions-to-outcomes

Cover these topics: - Lack of data on this generally because its hard to control for cause-to-effect - Quasi-experiments all kinds and the action2outcome project

The one-history problem

Cover: - Relate back to single trajectory discussion in previous post - All ways around this make assumptions which might be broken

Concept drift is another problem

Cover: - Concept drift and how it affects this - Adaptive simulations