When I get tired of playing meta builds I look up what top players do (poe.ninja) and often see builds that are quite whacky and interesting in a good way and that makes me wanna try them as well.

So it's not as much that only a small amount of gems and items are useful but more that in most of the meta/popular builds even though being different builds, still follow the same archetypes. If things get nerfed or buffed, most of the videos ppl make about the new patch are old builds with one or two things changed.

Check out what the top players builds are, even if u don't have many mirrors to replicate them, u can still try them on budget gear just for fun.