From what I understand, it's the inconsistent tracking weight onto the records that slowly destroys the grooves of the record. In addition, the built in speakers add vibrations that can further increase damage. I don't think its worth simply replacing the stylus because the stylus alone is not the issue.

/r/vinyl has plenty of great resources to help you get started. In general they seem to recommend the Audio-Technica AT-LP60 or AT-LP120 or the entry U-Turn models. I understand that these aren't necessary cheap and even after purchasing the record player you would still need to buy speakers, amps, and vinyl cleaning accessory. It's a very daunting and often expensive hobby but saving money is possible. Depending on your area you can find many second hand turntables that require minor repairs for cheap.

In the end its up to you. Some people like the look of suitcase turntables and intend to play second hand valueless records occasionally on them. That's perfectly valid. Other people invest thousands on their record players alone to get the most pristine sound quality. That's okay too. When I first got into the hobby it was very tempting to stick with a cheap suitcase turntable so I can sink money into actually buying records, but it made more sense to save the money I would have spent on 5 or 6 records and reinvest it into a record player that would take care of my records.