How do simulations run on CPUs?
As a stepwise loop built from memory, threads and channels between threads, with separate processes reserved for trajectories that need not talk to each other. When we talk about ‘classical’ hardware here, we just mean standard CPUs.
On CPUs, simulation architectures may be constructed out of several ingredients. Loosely speaking, these are: memory, threads, channels between threads, processes and Inter-Process Communication (IPC).
All of these ingredients have their own tradeoffs in performance. But they are all useful in constructing the right simulation architectures to satisfy the right use cases.
One of the main simulation architectures people consider is ‘stepwise’. These are simulation architectures which evaluate the next state values for the system at each point in time, in turn.
Stepwise simulation architectures on CPUs are typically more performant when using memory, threads and channels between threads in the right combinations.
In contrast, processes, and IPC in particular, are typically more useful when we consider scaling computations in parallel across multiple non-interacting simulation trajectories (which don’t need much IPC). This is because IPC comes with more performance limitations.