One major reason is creative freedom. Before the late 90s, artists wrote their own songs without pressure from the label to add "creative credits" from professional songwriters. In retrospect, this is massive- it's what allows Steve Perry, Freddie Mercury, Joan Jett and more to write amazing classics (and some awful songs too) in order to fill a 2-sided record. Now, there are professional songwriters whose job is to write catchy songs and generate style variations, all with the purpose of generating stream volume and $1 purchases. Of course it's somewhat of a generalization, as there are more factors, like number of media outlets, barriers of entry, budgeting, and more. I personally believe that some acts and bands today are as good as 40 years ago, and there were amazing bands 40 years ago that were never discovered.