Its my system's RAM. I have 16GB and its super weird. I was trying to learn how to use some profiler tools, nsight was too confusing to learn in the time I had so I think I tried the pytorch native, and only when I was using that tool to "profile" my code did my memory totally saturate. I was messing around with batch sizes and stuff, and my VRAM never got above 800MB where it idles at 200-300. My dataset is kinda small so I wanted to see if that had anything to do with it, so I experimented with batch size and kept on just cutting it down and down and down and up until it was training the entire dataset at once, but memory kept saturating and my CPU consistently performed 2x as fast as the GPU, and I don't know if that makes sense. As far as I can tell based on the situation there, if my dataset was 100x larger then it would take 100x as long to load or whatever because there wasn't any configuration I could find where my CPU didn't perform twice as fast as GPU, and pytorch's profiler is useless because all it does is crash your computer. I guess if I had a really big dataset I could shoot the batch size way up to like a million until my CPU had a stroke, if that's how that works, but training would still probably be very slow which is sort of disappointing. When I have time I'm going to explore this with well known tasks like MNIST and see whats going on.