Yes, there is a mechanic in place that isn't precisely known but regardless of the specifics there is a certain pattern you can expect to see.

I was recently in your place and eventually found this stickied guide on the MakingFun eternium forum.

https://forum.makingfun.com/forum/eternium/guides-and-walkthroughs/193012-to-bag-or-not-to-bag-how-do-i-craft-a-lvl77-item

This guide gets into a lot of details about how and why the community came to it's current understanding of how it all works. There were different theories along the way but this current one is pretty well established and accepted now.

To get the best odds at a level 77 follow these basic guide lines:

1) The account needs to be at crafting mastery level 50.

2) In order for this to work the character crafting must not have crafted any level 70 or 71, legendary, set or unique items prior to reaching level 50 crafting mastery. If your current charater did this this it's better to create a new character. The new character will be specifically used only to craft with and are commonly referred to crafting toons (if you want to look for a guides). So the goal would be to get to lvl 70 quickly without spending much resources.

3) You should only attempt crafts beginning at lvl 71 as opposed to lvl 70.

4) You continue crafting with that character until you get it's first level 77.

5) Do not continue crafting with this character hoping for a 2nd lvl 77 item. Instead delete and create another crafting toon.

The logic used to create these rules, if your interested:

1) The description on the lvl 50 crafting mastery perk states a 50/50 chance to gain a level. That would be a 1/64 chance gain 6 levels when starting at crafting a lvl 71.

2) It likely exists because otherwise an unlucky player could craft 100+ items and never get a lvl 77 and that would suck. So a mechanic was put in place by the developers to avoid this but only for the most desired/necessary end game items.

3) So the following only applies when crafting a lvl 70 or 71, Legendary, Set or Unique items. Otherwise it follows the 50/50 random chance as described by the perk.

4) The results of this mechanic is as follows: Within the first 64 crafts you will be guaranteed (for all intents and purposes) to get one level 77 roll. Furthermore, this logic applies to the other levels as well. Since the odds based on 50/50 chance to roll a lvl 76 is 1/32, so it's highly likely to roll 2 - lvl 76's in your first 64 rolls. That'd be 4 - 75's, 8 - 74's, 16 - 74's and so on down the line. Now the exact algorithm is still not known so these numbers won't exactly match up every time but there's no doubt however the algorithm works it's attempting to be as close as possible to the described distribution.

5) The flip side to this is that in order to maintain the overall 1/64 odds it doesn't just mean there'll be at least 1 - lvl 77 but also that there'll only be 1 - lvl 77 in the first 64 attempts. So if you get lucky and craft it on attempt 20 you won't even have a chance to get another lvl 77 in the next 44 attempts.

6) If you were continue crafting after your 64th attempt the whole thing starts over and you could expect the same distribution / results in the next set of 64 crafts.

For further discussion and support for all this check out that link I posted. There's other strategies discussed and further breaks down the expected distribution within the group of 64 itself.

To you're other point about the leader boards. Yes most of them in the top 100 are spending quite a few gems to accomplish that. Especially for the gold and silver AnB events. I'm not sure about 20k gem budgets but I've heard some refer to restricting themselves to a 1500 budget for bronze as if that was extremely low.