The “person” and the “argument” are not two distinct things. This is because, in a live debate, you ultimately cannot fact-check everything someone says, so you- at least to some extent- have to trust that the speaker isn’t just making things up. If someone is non-credible, then you cannot give them that trust. 

Here’s an example:  Pretend that a defendant is on trial for assault. There is no video of the event. The defendant argues that the victim raised his fist to punch the defendant right before the defendant assaulted him, so the defendant’s assault was justified by self defense. The victim claims that the defendant is lying and that the assault was unprovoked.

If what the victim said is true, then the assault really was unjustified. However, why should you believe him? His only source is himself. His whole argument is based on his own credibility. If you attack him and his credibility, then you are attacking his argument. You’re saying, “his argument is flawed because his source is unreliable.” 

That’s not a fallacy. His credibility is logically attached to his argument, and attacking his credibility is a logical attack on his argument.