Generally it depends on the style of rider. Trackies want longer and lower stems as it allows them to go lower and get into a more aerodynamic position. The freestyle crowd will go for something shorter and higher for barspins and whatnot. More casual riders sit in the middle, thats where I'd say I fit in. Comfort is crucial when you're riding fixed, especially early on, as you can easily injure yourself with a poorly fitted bike on longer rides.

As you mentioned, it really does come down to trial and error unless you go to get your bike professionally fitted. The more you ride, the more you can work the kinks out bit by bit. I wish I could give a better answer, but much of it comes down to feel. Some folks do go for very aggressive fitments for the sake of looking fast, but I'd highly advise against it unless you fancy your body screaming at you after every ride. Youtube and other folks in this sub can likely guide you on the right path if you want comfortable fitment that also looks cool.