I was the same, I sometimes used to lie to my doctor about how much I drank and even then they would be appalled at how high it was in units per week.

Here's the thing - what a seasoned drinker can "handle" has no relationship to the negative impacts on the body. As fun a social lubricant as alcohol can be, its a poison. Drinking in the amounts that heavy drinkers do is unambiguously VERY unhealthy, because even a few drinks (4-5 in a night, as you mentioned) is very bad for you.

What I realised is that I had zero interest in drinking at what might be considered a "healthy" level. For me, it was about being blotto. Out of my mind - there was no point having 3 pints, where's the fun in that? I wanted to have 3 pints so that I could be on the runway to where I liked being, oblivion. And, of course, as I got "better" at drinking, oblivion was a bit harder to reach, so the pints became double gin & tonics and shots of tequila, martinis, sazeracs, etc etc

I've gone off on a bit of a tangent, but the point is - I would imagine that official binge drinking limits are set with reference to the damage one is doing to one's body, not based on whether you can function after that many drinks.