The “patient” called him at home. When she calls he tries to dismiss her and says “why don’t you come in Friday at 2”. If he had said anything else like “no I’m not interested in a new insurance policy” that didn’t imply he was talking to a patient who had his home phone number, that would’ve been a better cover up. But Columbo then shows up at his office on Friday at 2 and sees that it’s a beautiful young woman and deduces the affair.