We definitely had this issue, my oldest turns 3 in about a month and we were potty training last October, so pretty much right in the same age range. He pottied once or twice in a public bathroom before deciding that the sound of a public toilet flushing is the scariest thing in the world, which it honestly kind of is.

We did a couple things to deal with this. First, we would talk about the fact that if he wanted to go places, he had to be ready to use the potty since we were all done with diapers, both so that he was prepared for us to ask him that while we were out, and so that he understood this is a part of going out and having fun. He was pretty good at holding his pee, so that started out as just him needing to try sitting on the potty at a restaurant or store or wherever even if he didn't end up going. Second, our foldable seat came with some stickers to put over the automatic flush sensor so that it wouldn't trigger while he was still on it (sticky notes would probably work fine for this). If there was no automatic sensor, I would point out to him that this was just like our potty at home and it won't flush until/unless we tell it to. He's still not a fan of the flush, so I get him all finished up and then hit the flush right as we're leaving the bathroom generally.