When does specialised hardware help?
When a batch of timesteps can be evaluated as dense arithmetic and fits in the device’s memory, so that the time saved outweighs the time spent moving data to and from the CPU. From the perspective of standard CPUs, batch simulation architectures are often designed to evaluate segments of the simulation timeline using specialised classical hardware.
When we talk about ‘specialised classical’ hardware here, we mean GPUs, TPUs, IPUs and other specialised processors based on classical computing principles (as opposed to quantum processors).
This architecture can be used to reduce the overall processing time taken to complete a simulation run relative to a stepwise equivalent, but there are tradeoffs which mean this isn’t always efficient.
GPUs, TPUs, IPUs, etc. all have their limitations. For example, GPUs and TPUs are highly optimised for dense arithmetic operations but struggle with branching control flow. IPUs offer more flexibility for irregular compute patterns and sparse operations, though they still prioritise throughput over the complex sequential logic that CPUs handle well.
So there are basically certain types of simulation algorithm that can be written that GPUs, TPUs, IPUs, etc. are not well-suited to reducing the overall processing time for.
In addition, this specialised hardware typically requires data transfer to/from CPU memory (at the very least for initialisation and final results), which also takes processing time.
So, when deciding on the number of timesteps a batch simulation architecture should use for the best performance, software engineers must take into account:
- the available memory of their specialised hardware
- the memory requirements for their simulation state partition histories
- the overall number of timesteps they need to perform
- the implications this has on the number of I/O operations needed to interact with CPU memory
- and the implications this has on reducing the overall processing time, given the specialised hardware.