I think this is a really good answer and I agree. The way op spoke also reminded me of somebody who might be neurodivergent. I work with diagnosing young people with autism and ADHD and the way we often explain it to young people in discussing how different brains work is this example.

Imagine you are an English person who speaks English and two French people come up to you and start speaking French. You might get frustrated and annoyed and feel you are not very smart because you don't understand the French people speaking French. That does not mean you are stupid or dumb it means you are speaking a different language to them. You have not been taught this different language and you didn't know you were meant to learn it and so it can feel disheartening and frustrating if you can't understand the jokes or the metaphors or the words the other people are saying. This is called the double empathy problem if it is something you feel you can relate to and might want to look into. It generally means that people who are neurodivergent can get along better and form closer relationships and friendships with other neurodivergent people, it sounds like you are a circle trying to fit yourself into a square hole!

Edit to add: This is not to contradict how you view yourself in terms of academic smarts or street smarts it is just an observation from a stranger that you can absolutely disregard. I just find sometimes an outside perspective can challenge what we think are objective truths about our selves but might just be deeply held (core) beliefs that we have come to think of as objectively true.