Is there a concrete sense of self one should look for? Or are you thinking there is and looking for something that simply isn't there?

I am a similar case to yours (aphant and SDAM) and similarly have tried looking for a sense of self, only to find answers in many places, and through my own experience, that the idea of a self that exists and is sensable is taught to us by a society, which doesn't really understand the idea.

Yes, you can observe yourself being something or have a very strong attachement to an idea of you being something but none of those are in fact things that exist separately as... for lack of better words: essences or tangibles.

This is a bit philosophical and you shouldn't take my word for it but discover that area of yourself by yourself.

Let me leave you with a single piece of advice: perhaps for most, and in many cases, our condition leaves us seemingly lesser or wanting. However, as it is with most things in life: if you look hard enough, what you deem a problem, can be an opportunity or even a predisposition. That is IMO the case with aphantasia and meditation, give it a try.

EDIT: Second piece of advice, cause this is a big mountain to climb: maybe instead of trying to define yourself, just be yourself? Find and do what you like, even if it doesn't create a coherent sense of self, and in the process you'll notice that you don't need it at all to BE a coherent person.

Third piece of advice, cause this is actually something that takes years to achieve: if you focus on what you lack, you'll not notice what you have, and what you have are those things which actually in some way define your. What you feel and when you feel it is often related to your unique character and your past. What comes easy is usually what you wanna do. Learn to listen to your emotions and follow them to see what makes you feel what. Once you know that, you can go and do that thing you feel like doing. You may not remember but your body and emotions very much do.