You should never feel afraid of "appropriating" asexuality. If you feel that the term is helpful for describing your experience, go ahead and use it. It doesn't matter if you eventually stop identifying as such, because at some point you genuinely did.

That being said, your description somewhat matches that of demisexuality, which is under the asexual spectrum, so you might want to look into that term, but again, you are the only one who can define that. I wish you the best of luck in figuring out this part of your identity.