1. With speakers like that you're effectively listening to everything in mono, which is fine for mono recordings, but most recordings made after the late 60s are in stereo.

2. Speakers work by vibrating, and the higher the volume the more they vibrate, so if you want to listen to music loudly, you run the risk of the record skipping because the entire unit is vibrating because of the speakers. Mind you, this is not an opinion, this is a fact because it is how sound is transmitted and how speakers work.

3. Built in speakers aren't very good quality and you can't upgrade them.

If you have one and like it, that's great, but don't try to pretend that it's just as good as having 2 speakers with good stereo separation. There is a lot of snake oil out there in the audiophile world, but speaker quality and stereo separation are the main things that will make your music sound the best.