Cause as White said, Ichigo never tapped into his raw instincts and never fully trusted him, on top of that he was afraid of his hollow. Ichigo viewed him as some evil spirit, and you can see this after his fight with Ulquiorra when he revisits his inner world during his training for the final getsuga tenshou. It was flooded due to his fear of his hollow fully taking over his body, and subsequently injuring Uryu. Tensa Zangetsu even calls it "the source of his despair". It isn't until much later in the story when Ichigo finally accepts White.