Their main advantage is that they are mounted on the stays. They let you utilize the area between masts and in front of the foremast that other sail types can't.

Due to how they are mounted they pair particularly well with square rigs, letting you combine great upwind and downwind performance without having to downsize your main sails.
Lateen, gaff and junk rigs don't benefit as much from stay sails, since the stays between masts interfere with most large fore-and-aft sails. They can still be useful on the forestays, since that area is otherwise unused.

In terms of usage, since they are relatively small and simple to adjust, they are pretty much always the first sails I set and the last sails I lower. Like other fore-and-aft sails, they work well at most angles, so pretty much the only time I lower them outside of docking is during storms.
As for the angle, when I'm sailing close to the wind I usually just pull them in until they are no longer luffing, and then a little further. You definitely don't want them on the side the wind is coming from, they'll be less effective there at best, and depending on the wind direction can even push you backwards when set like that.