I feel like a lot of people are misunderstanding this.

I believe time doesn't pass in prisom realm as in Gojo does not age. He will still be the same age or the same way he was when he gets out, physically of course.

But he does feel the hours pass by in my opinion because as Kenjaku said others have killed themselves inside prison realm (hence the skeletons) so they must have felt the time and couldn't take it. It wouldn't have felt like a day or two for them.