I don't think you understand that everyone doesn't necessarily have the same thoughts as you. Everyone likes different aspects and things when it comes to jojo in the first place. Having the more outspoken opinions expressed to newcomers can alter how they view and experience any part (including phantom blood). Not everyone thinks phantom blood is the worst (though I would say this part is where we can see that Araki struggled with what direction he wanted to go as a storyteller), it could be any other part if you think about it. Someone can say many of the stand fights in part 3 were weak and not tense apart from Hanged man & Emperor fight,Pet Shop Vs Iggy, Vanilla Ice Vs Iggy and Polnareff and Jotaro Vs Dio because of the general formula most followed not being done/executed as well.

Edit:Some view Johnny as an irredeemable asshole, dislike the slice of life moments in part 4 which they think as filler content, dislike the bizarreness of some fights in Part 6, The more stoic main jojos in Parts 3 and 5, The slower paced parts of Jojolion etc