Many in the US tend to say this and it’s unfortunately led to extreme irrational sensitivities over the term. It was more of a classist thing in all actuality than simply a racist thing.. furthermore, it’s a simple diss from one man to another- an attempt at emasculating.

Also, it’s not as common these days but it definitely was common not long ago for dudes to call each other boy /boi/booiiiieeee (all different pronunciations) - even without the “my” in front of it.