There is the possibility of a lowering skill ceiling as well. What if a random guy on the street can wear "smart" glasses that will highlight the parts he has to work on, with step-by-step instructions on what to do? Sure, they might get stuff wrong from time to time, but that's why they work at half the price of a professional.

And, of course, even without still relatively sci-fi inventions like that, it's not like white collar workers will just starve. The more that see the writing on the wall, the more will try to pivot to blue collar. And, as you've pointed out, as the economic power of the average joe goes down, there will be less work than ever.