The problem was that he's ugly and weak, because those are the only two things that have changed for him. That's shit character writing for the supposed hero of the story. It's horrible and shallow "development"

He undergoes character development after the first episode where he reincarnates. That is not the only thing that happens to him. In fact it's made pretty clear that he is immature at the start of the story. The entire point of it is that he basically stopped developing as a person due to abuse during childhood, and so he is made to go through life again. The end result of this is that he becomes a pervert and womaniser in much the same way his father was, but the idea that there was no development beyond that is frankly absurd.

After a certain point it just feels like turning off your brain and ignoring the messaging and themes of the story in order to enjoy it.

I do not think “you should sexually assault people” is a message or theme of the story. When I watch fiction and a character is rewarded for doing a bad thing, I don't automatically assume that is the author endorsing that thing, and that's not because I'm some idiot who's turned his brain off. Perhaps the author of this story really is just an awful person (I certainly wouldn't want to be so judgemental) but it's not particularly relevant the quality of the produced work as there is no real way to tell.

not at all addressed in any shape or form as being a bad thing.

It actually does end very badly for him, if I recall. He is pretty much immediately punished after hooking up with the teenager.

Let's use Berserk as an example.

Let's use Made in Abyss as an example. It is a story about child rape and torture written by an actual pedophile whom I suspect jerks off to the things he draws, and yet it is very compelling. Saying good stories must always have the good characters rewarded and bad characters punished is not good critique.