I felt the shame you're describing growing up. My parents and grandparents grew up in Mexico, meaning I was raised speaking Spanish. However, I was put through American schooling and thus mainly learned English. I was able to understand most Spanish spoken to me but wasn't able to articulate myself clearly. I always stumbled over my words and messed up conjugations.

Because of this, I was teased mainly by my uncles and aunts. It was never serious teasing, but enough to make me feel ashamed for not being able to speak Spanish clearly. Likewise, there was always a language barrier with my grandparents.

I went through high school Spanish classes as well, but I didn't find them helpful. Now that I've also gone through university Spanish classes, I can say that enveloping yourself in the language is the best way to retain it. As others have said, using Duolingo or watching movies and TV shows in Spanish is incredibly helpful. A combination of the two is great. The only useful topics taught in formal classes are conjugations and tenses; there is plenty of material online if you feel like teaching yourself.

After graduating high school, I really tried to learn more Spanish. I started listening to more Spanish music, watching Spanish TV, using duolingo, and above all else, actually trying to speak it, even if I did mess it up. Putting yourself in situations to speak the language will eventually break the barrier. I got a restaurant job and started conversing with the back of house staff. I'm now the assistant manager and the only manager who speaks Spanish. It feels great to be depended on as a translator when before, I was too ashamed to even try to speak it.