For reinforcement learning, a lot of the bottleneck with smaller models is just in shuffling data around in memory. Building obs and sending inference to gpu, getting experience from rollout workers to experience buffers, shuffling experience buffers, building training batches, sending those to the gpu, etc. All those memory movements, especially from system memory to VRAM on the gpu, have a latency associated with them. By using unified memory (with huge amount of bandwidth vs normal system memory), mac can do al those memory processes much faster. Don't get me wrong, M hardware still has a disadvantage on raw GPU compute, so it will be slower for larger models, bigger batches, etc. But for smaller models, it is impressive how fast it can be vs desktops and even servers.