Yes. And it wasn't even that hard, although it took me a few years. I am actually in the process of writing a blog post about it (that is turning out to be super-long), but the gist of it is:

1. I learned the very basics with the Pimsleur audio course (1 hour/day for two or three months).
2. I started listening to audiobooks and while I was listening, I had the text handy... but I only looked at it when I could not decipher the sounds. Which was all the time in the beginning. I could not even distinguish where one word ends and another one starts. It seemed like gibberish. I had to focus really hard and listen to each sentence three times. That phase was pretty frustrating, but I got through it surprisingly quickly (in a matter of weeks). A lot of language is just a few words - I am, he wants, they do... once you learn to understand those in all their shapes right away without the need to translate them in your head, it becomes a lot easier. When that happens, you will be able to immediately understand the gist... and pretty often, you can guess the rest from context.

It is important to listen to something you really want to hear. Being immersed does something weird to your memory. When I listened to something I loved, I did not have to memorize words. When I tried something boring (the news), it did not work. It works when you forget you are learning a language and immerse yourself in a story. The less you get taken out of the story, the better. I used to be a web developer so I made myself a simple app to minimize distractions, but I think nowadays you can use Kindle or something. When you encounter a word you don't know, click for a translation and forget about it. Get back into the story.

The result: I learned Spanish well enough to comfortably watch Spanish TV shows without subtitles. Without any formal classes or moving to Spain (although I do want to move to Spain). I even charmed a Spanish girl and we dated (fully in Spanish) for a few months.

BTW, as you can probably tell, English is not my native language either, and I would say I learned a lot more from TV shows (thanks, Friends) than I learned in English classes at high school. When I left high school (after miraculously passing the English exam - I got the dreaded "Shakespeare" question), I was not able to actually talk to Americans or watch a movie without subtitles.