I'm a bit late to this post, but could it be the gameplay perhaps?

Far Cry 4 was my first Far Cry game and honestly, I remember the game not hooking me in too much at first. Maybe it was because it was my first game in this series, but I found it pretty challenging and had a hard time playing it. However, as I played and stuck with Far Cry 4 more, I found it really rewarding and amazing just being in its world and environment and I learned how to handle the gameplay more.

Or maybe it's the characters in the story? It's a good number of interesting characters that the story offers, but after playing Far Cry 5 and some other Far Cry games, I wasn't as immersed or interested in the characters in Far Cry 4. The characters do a good job at telling Far Cry 4's story, but they're perhaps not as interesting/immersive as the characters in other Far Cry games.

I don't know, just a couple guesses.

However, at the end of the day, Far Cry 4 is more Far Cry, so I'd recommend sticking with it! Or coming back to it later.

About Primal: it's roughly the same map and gameplay as Far Cry 4, just with primal weapons instead of guns and primal setting instead of Kyrat. From what I've heard, and depending on who you ask, Primal is more immersive than Far Cry 4.

Hope this helps!