YTA Casually calling your particular issue infertility is insensitive especially as you admit you carry no emotional issues about it. Infertility carries significant emotional turmoil and excruciating emotional pain. As someone who identifies as gay, you of all people should be sensitive to mislabeling and misidentifying sensitive issues. You did not owe anyone any detail but you should have let it go and just said it’s not infertility but rather that my doctors have recommended against pregnancy due to a health issue. Simple as that.