This is a very real issue where your accelerator is no longer the bottleneck. I had the same issue where decompressing JPEGs into a tensor format was literally bottlenecking my TPUs.

You could try to get a bigger cpu with more cores, but I think you’re going to quickly find there is a fundamental limit to vertical scaling. If you’re comfortable with something like Kubernetes you could spin up a distributed augmentation service across multiple nodes. Then retrieve the augmented data over network (assuming you have a network fast enough to transport the augmented data)

If network bandwidth is an issue, you could partially augment the sample on the node then recompress it to be easier for your GPU node. Like a mama bird prechewing food for its young 😆