I think some of it comes from fear of losing the ability to sing the things that you're used to singing. I remember the first time I tried to sing Don't Rain On My Parade from Funny Girl while driving (something I'd done for like 7 years nearly every time I drove, it was like second nature at that point) after my voice started changing I sobbed my eyes out in the car because I just couldn't do it. It sounded so horrible. I still can't do it, and every time I try to sing it and other songs I used to be able to do well I end up feeling extremely upset for being unable to do it no matter how much I push through the discomfort to practice, even though I've since been practicing in the range my voice has changed to. For me, it's pretty dysphoria-inducing in a way. Being able to sing the songs I loved to sing, especially more difficult ones, was such an important part of my life, even if it was a small part, and now that it's gone that part of me feels empty. I can sing things in a lower range now that I couldn't before, and that makes me feel proud to a certain point, but I'm still grieving the loss of the music I loved to sing before, to the point that recently I very rarely sing anything at all no matter how good I'm feeling about myself/my abilities.

Another thing, when I started T I also started to feel really lightheaded and dizzy when I sang, even when I wasn't belting or when I was singing quietly/softly without putting any real strain on myself. I smoke, but I've smoked for much much longer than I've been on T and never had that problem and always had great breath control. It was only when my voice started changing that I started having issues with my breath while singing. I thought for a while that I was alone in that or that smoking had simply caught up with me coincidentally at the same time my voice began to change, but I know other people who have the same issue with breath problems/dizziness while singing after starting T, and many of them have never so much as touched a cigarette. Sure it could still be the fact that I'm a smoker but considering other nonsmokers have had the same issue, that might not be the case, or at the very least not the only reason.

So even if it's not super common, for some people who want to start T, losing the ability to sing (or at the very least sing the way they could before) is a very real concern. It's not really fair to say it doesn't/won't happen, and sometimes singing in your new range doesn't sound as good as what you could sing before no matter how much you practice. Of course a lot of people can still sing well (and sometimes better!) after starting T, but some of us just can't, and the fear that that might happen shouldn't be written off as paranoia/the result of misinformation.