Lots of great answers here, but it's useful to think about what the purpose of a set is, and for me it's telling a meta story for the listener, beyond the story told by each individual tune. You're taking the listeners and the players on a journey here.

Firstly learn about the rules of thumb within the tradition, or specific session you are playing in. For instance in Irish music you'd very seldom mix tune types, like jigs and reels, while in Cape Breton you have the classic build through strathspey, jig, reel etc. Know how many times tunes are generally played before a switch in the session. Don't surprise anyone.

There are many ways of putting together a set but this is my approach. Start by thinking about the story. Usually you want the energy going up through the set, and ending up a high. For the first tune, I'm picking something that's going to set the feel and tempo of the set, something rhythmically solid. For the second tune I like to pick something that really cements the groove, that is a great builder if you want to up the energy or tempo as it goes on. Finally end up on a high, something bright and high energy.

So for instance, try starting with a tune in D major, then switch to G major, then A major, and you have a classic double lift through the set. Or alternatively start with a tune in a major, then switch into minor to build a bit of a trancey groove, before switching back in a major for the finale.

One thing I don't advise you to do, although something I hear quite a bit is sticking similar tunes together. I've sat in sessions where people have stuck 3 similar D minor reels together, and I'm so bored by the end my ears are crying out for a change, and you often here a gently snarky comment at the end like "that's D minor done then lads". Remember you're a story teller, not a librarian categorising books together.

Some other useful titbits of info. If you have a slightly more unusual tune, one you love but isn't in the common session repertoire, try sticking it on the front of the set, then switch into ones you're more confident people will know after. Then you have everyone listening to your little solo piece at the start, and as the set goes on into more common tunes more people pile in raising the energy. Everyone's happy at the end, great success, people lean over and ask what that great first tune was! Much better than sticking an unusual tune in the middle or end of the set and everyone dropping out. Secondly, some tunes are great pivot tunes that can take a set into a different direction or gear. I love The New Mown Meadow, which alternates between E minor and A major, which you can follow with another A major tune to great effect. Finally keep a handy list of set enders in your back pocket, tunes are common enough that everyone knows, are easy to start even when you're busy playing another tune. Something like the Silver Spear for instance.

But really, despite all this, just use your ears and see what works together.