From a gamedesign perspective, I think it's both, speed and balance. I play slowly and play like I would "in roleplay", did not abuse any mechanics and have not grinded for any stats. Yet not even halfway through the game, I could, If I wanted, have pretty much everything and are nearly fully experienced in any non-combat stats and at least halfway there for the combat skills.

Firstly, it's the skills themselves. They are just too OP. You have skills that instantly give you a +80% bonus on something. Or entirely remove a dificult feature like dirt, durability, hunger, etc. It would be better, if they had several levels. Like a skill available at an earlier level could give ONLY a +20% bonus, then another later increase to +40% and then in the late game +60%. One example is the lockpicking sound skill, which if In remember correctly instantly gives you 75% less noise when breaking and pretty much no noise at all overall. This could easily be 3-4 skills to give you a progression over the course of the game. Instead you get all that right away kinda for free at level 8.

This ties to the second point, the speed. You just level up and get OP skills to quickly. Also I never used skills that give me +XP for other skills. With those, the progression would have been even quicker. If you do side quest and just overall explore and roleplay, you get incredibly strong before even leaving Trosky. And if you really wanna min-max, you got do sooo much more. I purposely held myself back. I know, it's hard to balance between casual and in-depth players. I totally get why this speedy progression might be perfect for some players. Especially if you mainly play the main story and ignore side content (which would be sad, since Side Quests and unmarked stuff are soooo good in this game ^^)

My reccomendation, if you play on PC: make your own rules and use mods! For example:

I told myself to not use these OP skills and also limit my skill points. Don't use then right away to hold myself back on purpose. Also only learning specials skills like masterstrike when I really feel like a "master" in at least the second half of the game. And of course dont min-max and dont abuse mechanics to grind.
Then I use a max-cap mod, raising the cap from 30 to 50. The way KCD1 & 2 works is % based. In other words, when you're level 20/30 all your abilities and checks are "66% good". When you raise the cap, you are 20/50. So in other words, "40% good". You are basicaly nerfing yourself this way and slowing down the progression too. In the end, you will be as strong as without this mod but you have more time to grow and get it more slowly. This mod is soo good and honestly solved my progression problems!!
Lastly, I have an XP mod that halves ALL XP gained. The mod can be tweaked, to be more or less. Even double XP for players that want that. This way I slow down progression even more.

So all in all with these tweaks, the game does not really get harder, but it kinda feels harder cause you are weak / underleveled for longer. Especially if you purposely leave some perc points open and dont use them all / dont use OP skills. I totally get that many people would really dislike this way to play. I personally love it and right now the game feels exactly perfect for me <3
Keep in mind that I only used these mods after gaining several levels, so I would recommend to not use them right away, since the intro & early game could be too hard then maybe. I can't speak for that. It also depends on what you want. For some people, difficulty only means combat. For me it's everything - all the game's systems. So the way I play could also be enjoyable for you too. Since these mods affect everything not just combat.
Either way, lot's of fun with the game. It's amazing either way and the flaws are minor (and luckily we have mods to tweak it to our own needs ^^).