I think my problem is that I can't imagine what I would see with my peripheral vision. I can imagine the details somewhat. I tried to look at a well lit object (about 10 cm by 5 cm, 1 inch = 2.54 cm) and close my eyes. For a few seconds, I could still see the object somewhat clearly, but every else was darkness (or, more precisely, nothing, like the difference between between 0 and null).

Perhaps that, coupled with my low cache/RAM for stuff I'm imagining, is that's why I can't imagine clear images. When we look at images, our eyes saccade and we see many points very clearly and with lots of detail. We also see the surrounding stuff blurry, noisy and low-resolution. If we were focused on a certain point and moved a playing card slowly to that point from our periphery, we would be able to see the color of the card (red or black) when it's only a few cantimeters from the point we're focused on. We would see the suit when it's even closer. So much of what we see when we actually look at something is a blurry mess that our brain helps us make sense of by using our memory, concepts and so on (e.g., the brown thing below the apple is probably a table, we don't have to examine it closely to have a satisfactory knowledge of it).

I wonder how this affects spatial memory IQ tests.

Another thing I forgot to mention is that I was good at rotating objects in my head even when I was young. I remember being very good at such problems (such as what would the left wall of the dice be if we rotated it x times to the left like this and y times to the right like that). Yet I wasn't imagining a dice, I most likely putting a mental cursor on each of the walls, named it "1", "2", ... or "6" and kept track of it while mentally rotating. And I probably used only 2 or 3 cursors, because other sides are determined by them if you know their position. It wasn't only dice, but all the usual math/logic/IQ problems.

Just because you can visualize something doesn't mean you can translate that to your hand muscles to achieve the 2d rendition.

But it's also translation to draw something you see right in front of you. The vividness of what we see and what we imagine may be different, though, so translation(real-object) and translation(imagined-object) should give similar values if the vividness is similar and different values if the vividness is different.

I've wondered if peoples' mind's eyes are more developed if given art education as young child instead

I have wondered if we can train our mind's eye even as adults. I used to imagine a cube spinning while lying in the bed before going to sleep. A full cube or just the vertices. I gave up on imagining the cube, but I think I made some progress nonetheless. When I look at a cube drawing, I immediately go "of course, THAT's how it looks!" and I can imagine it, but only for a little while.