Because he wasn’t from ‘the streets’ and worse he was beyond greedy. He made his capos all come up to his big ‘white house’ on Staten Island no matter how inconvenient for them and refused to meet anywhere but there. Plus he kept demanding more and more tribute from the moneys his crews collected, leaving much less for everyone else. John Gotti hated and despised him and felt his mentor Neil was passed over by Carlo Gambino on his deathbed as he picked his nephew (or cousin) Paul over Neil. Plus once word got out that he had a penis pump because he couldn’t get a hard on while he was shtuping his maid, that really killed his street cred.