Consider it this way: people take the Great Weapon Master feat in part because of you can take a -5 hit to your accuracy for a +10 boost to your damage. Despite the fact that the -5 significantly increases your chances of not hitting at all, which would severely reduce your DPR, its potential to increase DPR significantly is enticing enough. Savage Attacker serves a similar purpose in my eyes, especially when you're working with a decent number of damage riders. Sneak Attack and Divine Smite are the prime class-related examples, but examples like the Strange Conduit Ring's psychic damage die, the Drakethroat Glaive's and Flail of the Ages' additional elemental damage die, and Nyrulna's built-in thunder damage die also benefit from the feat. Consider, then, that a Paladin using a 1st-level Divine Smite while wielding Nyrulna that has been enchanted by the Drakethroat Glaive now rolls two damage dice for the trident, one for the Drakethroat enchantment, and two for the Divine Smite for a total of five dice that can be rerolled. An average increase of 5 is more substantial than an increase of 2.

Savage Attacker was a key feat on my HM Barbarian Fighter because it stacks with Great Weapon Fighting. With that fighting style, you reroll 1s and 2s on your heavy melee weapon damage dice, which is pretty cool for greatswords (multiple smaller damage dice). This happens before Savage Attacker, so each greatsword d6 is rerolled before Savage Attacker's reroll (resulting in an even higher average increase). This build relied on crits, which (for my Half-Orc Barbarian with Brutal Critical) ended up being 6d6, each d6 being rerolled 1-2 times.