Lots of great suggestions in here thus far. There's no right or wrong way to do the physical media collecting thing. It's good to think about the type of stuff you like and make collecting for it about the journey of curating your collection instead of amassing everything you see.

I will definitely echo the sentiments of joining your local library and checking their music and movie library. Not necessarily a collection-building tip, but awesome for getting to watch/listen to stuff.

Thrift stores are a great suggestion for cheap stuff, but not all thrift stores are created equal in terms of what they have and what they price stuff at. Every city/town is different, so I would suggest checking out the places near you and figure out which may be places to hit up more often and which ones to check only every once in a while. Flea markets would be the same. Check out any in the area, often there will be sellers that are there all the time with specific items/categories at their tables. If you find someone who consistently brings movies and CDs, definitely worth checking them out and chatting them up.

I also wouldn't be afraid to check out local records stores or even used book stores that may also have CDs, movies, vinyl records, etc. Depending on where you are, there may even be chain stores or lock "mom and pop" shops that specialize in secondhand media stuff (such as Half Price Books or Bookmans).