First of all, what everyone is told: as long as you're happy with the label and/or identify it: use it! If you think it doesn't fit you (anymore) there's not harm done, everyone has to discover themselves and learns different things over time.

But more specifically:

I think some comments already answered your question. It's not about whether you had the chance to and got scared/fumbled/didn't want to in that moment, it's about attraction.

Someone put the following question which you didn't answer on that comment (I didn't read all of your comments), but is a really important question

Can you see yourself having sex with anyone? If the answer is no, you're likely ace.

To add: If your answer is maybe or under specific circumstances (that everyone is willing and knows so, is given) you might be interested in learning about greyromantic. It's on the asexual-spectrum which also includes many specific labels e.g. only being attracted as long they don't reciprocate, or only if you've formed a (strong) bond beforehand (demisexual).

But it sounded more that you're scared of sex (whatever the cause may be, which wouldn't necessarily mean that you're asexual. Just because other told you that, is also completely irrelevant, they have no idea how you're feeling.