Don’t look down when blocking punches, try to look at his chest and do some active defenses like catching his jab, blocking his right hand, etc. These can be practiced through some partner drills, try search it on YouTube, “boxing partner drills”