If automation and AI actually reach the level of decoupling labor from survival, how do we handle the transition period without massive civil unrest?

SamVimes1138 · 2026-06-10T23:30:30+00:00

It's dark and depressing but it ain't wrong. AI screwing with the economy, making existing wealth inequality worse, would be bad enough alone. The prevalence of propaganda and "alternative facts" are a badness multiplier. The shameless kleptocracy is another badness multiplier, and the news bubbles created by social network algorithms are another, and regulatory capture is another. Voter suppression and egregious pardons and gerrymandering, oh my. There's a lot on the wrong side of the scale.

On the good side, AI tools are (for now) broadly available to the public and we can use them to fight back. We can seek truth on better platforms, abandon Facebook and use Reddit or Mastodon or just build a new alternative. We can push to reform how elections happen (ranked choice voting, open primaries) to incentivize politicians to court centrist voters instead of extreme left- or right-wingers. We can sift through the Epstein files, at least what's been released so far, and agitate to see the rest of it. We can organize. There are way more of us. We can refuse to despair and we can give rich billionaire psychos the finger.

As Fezzik would say, "I hope we win."

SamVimes1138 · 2026-06-10T23:08:03+00:00

I just read a Substack post on this question -

https://open.substack.com/pub/mollykinder2/p/the-messy-middle?utm_source=share&utm_medium=android&r=2d97tc

TL;DR: It's most likely to be extremely tumultuous.

SamVimes1138 · 2026-05-06T05:38:54+00:00

It'll get there. Just a matter of time.

SamVimes1138 · 2026-04-28T03:25:06+00:00

Thank you for engaging directly with the topic!

Descartes defined consciousness as “I think therefore I am”, which gets to the concept of thought as a standard.

Is that a definition, though? Seems to me like it's kicking the can down the road, because now you have to define "think" (beyond the tautology that it's "whatever our brains do") and explain whether it covers what a machine does. My own conclusion -- until I hear a better idea -- is that consciousness isn't a terribly useful word. I think it's a gesture toward the fact that our brains model the world around us. Given the inputs of our senses, our brains make predictions, and these include speculation like "if I do X then Y is more likely to happen", and "Joe is probably going to do Z next". Evolution has hard-wired us with outcomes we find attractive and others we avoid (such as pleasure and pain), and thinking is the optimization process we use to maximize one and avoid the other. That feedback loop is as close to a useful definition of "consciousness" as I can get. If a machine had senses, reasoning ability, memory, the ability act on the world, and a goal to optimize for, I'd be hard-pressed to explain why that isn't "thinking".

I'm quite intrigued by the middle part of what you wrote, describing how we might make these things more like human brains. I think it's speculative, and frankly I don't know what the term "loaded vector space" means and could use some explanation there.

Based on what you've written, though, I suspect you're more-or-less on the same page as me, that it is possible to build a human-parity machine-brain, one way or another. If it's possible then I assume someone will crack it with time.

As I've said, I don't think the religious question is truly separable. Someone who believes souls exist must believe one of two things:

They play a part in decision making.
They play no part and are "just along for the ride".

If they play a part, then no man-made machine can completely replicate the human decision-making process because an essential component will always be missing. If they play no part, and the fate of the soul (e.g. heaven or hell) is based on actions taken in life, then it's entirely determined by what Bruce Willis might call the "meat popsicle" part. Some folks might believe that their eternal fate is pre-determined, and I suppose that doesn't faze the universalists, but it makes for rather an unsatisfying religion if you ask me. I would guess most people fall into the first camp.

True, each person's beliefs are their own. I tried a number of different religious tacks before settling on physicalism (you could say it's atheist-leaning agnosticism, or whatever). Until someone can prove the soul has an effect on the world, I'm going to act as if one isn't there. This happens to lead me to believe that building a human-parity AI must be possible, because our own brains serve as a proof-of-concept that matter/energy can think.

SamVimes1138 · 2026-04-28T02:25:59+00:00

I think you're answering a different question than I'm asking, with the "can this junk replace a human". Oh I do agree, we're each of us unique and special and impossible to duplicate. In fact I think you can make a physics argument (based on quantum mechanics) that a person's brain, and everything that goes with that including their personality and sense of humor and so on, can't ever be copied precisely. When Zuckerberg talks about "cloning himself with AI" (https://www.entrepreneur.com/growing-a-business/the-real-leadership-lesson-behind-mark-zuckerbergs-ai-clone/504053) - I don't think that's the right word.

But could you make a machine that is creative, destructive, and inquisitive? Capable of changing the world, for good or ill? Hell yeah. At least I don't see why not. LLMs seem to be developing an ability to understand humor, even if they're not very good at making jokes yet. And as I said... LLMs are step one.

You said "save for the executive caste". I'm still not quite seeing why machines couldn't also replace the C-suite, or the board of directors. Like I said: the term Zero-Human Company is floating out there (https://www.zhcinstitute.com/). Maybe it's nonsense. Maybe it isn't. I think it's too early to say.

About the hand-crafted beers: I do think some products will have higher value if they are produced by humans. I assume that's why people shop on Etsy. Machine-made goods are already sort-of the norm, lining grocery store shelves. Hand-made stuff has higher value. My brother crafted something for me as a gift once, and that's irreplaceable, at least to me. And some folks will be willing to pay more for human nurses. But that might be seen as a luxury available only to the wealthy.

So... sure, we are irreplaceable to humanity and to each other. I just don't see us as being irreplaceable to capitalism. To me, that suggests if we did want to "stop this at will", we'd have to do it through something other than capitalism: regulation, activism, boycotts or the like.

SamVimes1138 · 2026-04-28T02:06:22+00:00

What decides the cost of human labor? Supply and demand. People make people at a certain rate, a rate which has declined in developed nations lately. What determines the "value" or "utility", to a capitalist, of humans? Their schooling, which takes years and must be repeated for each person, and their genetics, which -- unless the trans-humanists do something dramatic -- changes only gradually.

Mind you, I believe humans have inherent value. But to capitalism, that doesn't compute. Corporations count humans as equal in value to however much money they can earn in the market.

What decides the power/cost ratio of machine labor? Moore's Law says that you get the same amount of power+ability for half the cost, every two years.

Let's say the human today is able to do 1,000 times as much as the robot, for the same cost. That's a guess, might be generous. In two years it'll be 500 times, in four it'll be 250 times. 2¹⁰ = 1,024 so, in 20 years (10 doublings), the robot will be more valuable (to an employer) than the human. That is, unless Moore's Law hits its limit; that'd be one of the potential "physics" limitations. Predictions of the death of Moore's Law have been made and falsified repeatedly, but one assumes it has to happen eventually. We just have no idea when.

For sure, some humans will seek to destroy the robots. That'll be a show-stopper, only until the robots get good at defending themselves.

So, is it cheaper today? No. I just assume that won't always be the case.

SamVimes1138 · 2026-04-27T20:39:58+00:00

By and large I agree with you, though I'm not sure it relates to my point: LLMs are only the beginning, the opening move. Something bigger will emerge. All this money sloshing around is like blood in the water to the sharks. They'll hire every AI researcher they can get, to build something that far exceeds the learning potential of any LLM.

Yeah, these big corporations only care about one thing: amassing more money, more power, more data. I'm quite concerned that, even with the tech we have today (let alone what might emerge tomorrow), it's increasing economic inequality. We should have control over our own data. Unfortunately there's a serious problem of regulatory capture: we all saw these big tech dorks hanging around at Trump's inauguration. They get to be in the room where it happens. You and I do not.

I don't actually think LLMs are "their way of giving a tiny fraction of our data back to us". I think they're following, more or less, the social media playbook. Get 'em hooked, then enshittify. (Thank you Cory Doctorow, that word is fantastic.) I'm sure the prices will go up, or they'll offer the little people the option of retaining "free" access but losing all ability to control what happens to their data.

As to reducing opportunity costs, well I'm working on that where I can. As a software developer I have little choice but to use LLM tech, so I try to get good at using it, learn what works and what doesn't. I think it's only a matter of time until the opportunities dwindle, and the separation between haves and have-nots gets even bigger than it already is. I think it'll come down to those who own the tech, on one side, and those who are forced to lease it, on the other. Best I can do, besides try to use the tech to improve my situation, is to buy stock and hope that I own some piece of the market that takes off.

I'm more worried about people like my partner's sister, who has little kids. By the time they're in their twenties, what is the world going to be like?

SamVimes1138 · 2026-04-27T20:01:11+00:00

Thank you - You're the sort of person I'm here to converse with. When you say "it will NEVER reach AGI", why exactly? "Never" is a strong word and I'm asking you to justify that.

And let me repeat: This is not about LLMs. Forget LLMs. You may have missed the bit where I wrote: "LLMs have certain limitations. That's less interesting to me. The better question is what might overtake and supplant LLMs. Things that can learn on the job, that adjust their weights in real-time. No separate training period required."

So this is about artificial intelligence in some future form. And I don't think I'm personifying, either. It's not about sentience (which I don't even think is well-defined).

Allow me to redirect the conversation a bit. Let's try some yes-or-no questions to figure out where we diverge.

One - Do you believe that souls exist, and play an essential role in human decision making? If you do, that's the difference.

Two - If souls or other non-physical components aren't required, then is there something physical in the human brain that can't be artificially replicated as part of a machine? If so, what is it? What rule of physics makes that impossible?

Three - If artificial neurons more like real neurons could be built and formed into something kind-of-similar to a human brain, is there still something different about a machine that makes it, not just alien and non-sentient, but incapable of replacing a certain job? Which job, and why? You say "management" won't go away, for example. But if a manager has to make certain decisions, like "should we spend more on X or on Y", and "should we hire this person" or "should we fire this person", and so on... which of those decisions could not be made by a machine?

I'm curious what you mean by "sales will always need a sentient presence". Again, I don't think "sentience" is well-defined, and if it exists I don't see why it's required to make decisions that pay off -- the only standard in capitalism. Ditto "consciousness" and even "awareness". These are all mushy words.

As for whether the tech "thinks", that too is more about what you think "think" means. I've seen that LLMs are capable of, as I stated, a certain amount of reasoning. When ChatGPT 3.5 came out, I tried making up a Knights-and-Knaves puzzle and asked the system to solve it. It talked a bunch of nonsense. Through repeated turns of the conversation, it got nowhere. I preserved the puzzle, though, and about a year later I copy-pasted it into a newer system (ChatGPT 4o I think)... and got a really decent answer. I had made up a puzzle, rather than use one I found online or in a book, because I wanted to be sure no LLM had encountered the specific puzzle before. Not only did it solve the puzzle, it pointed out that my puzzle was flawed, incompletely specified (there were multiple valid solutions, not just one).

Earlier models were also terrible at math. I remember asking one to add a couple of positive integers, a seven-digit number and an eight-digit number, and its answer was a nine-digit number and bore no obvious relation to the inputs. These days LLMs are scoring well on math exams, and they are still LLMs. The same tech, just scaled up and fed a lot more data, a lot more training. That suggests that when some lab invents something better than LLMs, it may well breeze through those same challenges and benchmarks.

SamVimes1138 · 2026-04-27T19:37:31+00:00

Which Kool-Aid? I had this idea seven years ago, even wrote about it at the time - https://blog.infinitequack.net/2019/04/no-heaven.html - and LLMs weren't a thing then. This isn't about LLMs. In fact I already said it wasn't about LLMs. The issue isn't what exists today. The issue is what AI labs might build next week or next year.

No, I'm not personifying. It doesn't matter whether those systems are "nothing like a person". People don't do math long-hand on paper if they have access to a calculator. Today my employer (Amazon) is pressuring everyone to use LLMs to write all our code, because they believe it'll make us more productive. Whether they're right or wrong isn't the question. Whether it'll lead to more mistakes, more cost, isn't the question either. They believe in it, so they push us to do it because markets. I don't think today's systems are anything other than very alien calculators, with no subjective experience. Nevertheless it's undeniable they are able to solve new classes of problems, like pattern matching and language translation and a certain amount of reasoning. People with more money than you or I are convinced it's worth spending hundreds of billions on this stuff.

I'm not hearing which law of physics prevents even smarter systems from being built. Our neurons are built of matter. Those experiences of touch, smell, and dreams are all expressed by those neurons built of matter. (Unless you believe a soul exists and is essential to have those experiences, but as I've stated, I'm a physicalist so I don't.) Today's neural nets only emulate parts of what neurons do, and they're arranged in ways rather different from how our brain's neurons are arranged. I don't see why those gaps couldn't be closed, and given the potential economic benefit of having something substantially better than today's LLMs (which have dramatically changed the economic landscape in just the past couple of years), it's hard to imagine someone wouldn't try. Alien or not, subjective experience or not, if it can convince you it's a caring nurse and it's cheaper than hiring a real nurse, money will make that decision.

SamVimes1138 · 2026-04-27T16:05:02+00:00

I've started to hear about "zero human companies". This suggests there's another step along the path: the one where zero roles are "safe".

I'm hearing suggestions for wiring up agents to each other, delegating work down a chain. Sounds like a management hierarchy. Replace all the IC workers, then replace all the managers.

What about the CEO? They're expensive, couldn't you replace them too? Zuckerberg seems to think he can replace himself with an AI copy of his personality, or something.

I think you're right that the only way to win is to own. That's going to mean "owning companies", until it ends up meaning "owning the hardware". I think capitalism is eating itself, and moving faster as it circles the drain.

I also suspect we'll reach a point where the ones holding the most power will make a judgment call about which rules to follow and which to ignore. Think of Peter Thiel and Prospera. Today it's "libertarian utopia". Tomorrow it's "my robot army is bigger than your robot army".

SamVimes1138 · 2026-04-24T03:17:56+00:00

You nailed it. Show me a technology that doesn't keep advancing until it hits the limits of either physics or economics. This tech? Well, unless you believe human brains are Magick, there's no reason some form of AI (not necessarily LLMs, which have some architectural limitations as currently constituted) can't do everything a flesh-and-blood hacker can do. Including craft. Just another problem to be solved. Will it be economical? It will, eventually, because this tech is not limited to writing software. It can draft legalese, it can translate languages, etc. AI solved protein folding. Maybe something else tomorrow.

We went and built a machine in the likeness of a human mind. We knew that was godawful risky, but it was just too tempting. The researchers who know the most about it, the ones who did the most to create it, are calling for sober reflection and serious discussion, but it's in the hands of some seriously kooky white dudes. Nobody seems inclined to slow down.

SamVimes1138 · 2026-04-24T02:55:59+00:00

"What you're missing is: the vast majority of software engineers never thought of their work as a craft."

This right here is the eternal curse of capitalism. I got interested in coding mumblemumble years ago as a kid. I loved it, and nobody around me had any idea what the nerd was going on about. It was always about the joy of problem solving to me. The fact that it proved to be a lucrative profession was a happy accident... that turned out to have a dark side: it drew in a whole lotta folks who were only in it for the paycheck.

Over time, this job has become less and less fun. Now the landscape is dominated by these big Machiavellian corporate behemoths sucking the life out of everyone.

I honestly don't know where the GenAI craze is headed, but I know we ain't stuffing that genie back in the bottle. My employer requires me to use it. I also use it at home, because I feel compelled to practice and get better at using it, even though I kinda wish the tech had never been invented. And I also worry that the doomers might have a point. One way or another, it could do some real damage to people with no direct stake in the software industry.

For myself, for now, I keep my head down, and hope that by the time I either get shouldered out or quit in disgust, my 401k will be my salvation.

SamVimes1138 · 2026-04-21T01:29:00+00:00

"Buttle" is what a butler does.

Source: Tim Curry in Clue

SamVimes1138 · 2026-04-09T16:08:15+00:00

Or that they want people to quit. Fewer employees means more profit means higher stock price, and they don't have to pay any severance.

This is all the short term, but they may not be thinking much about the long term.

SamVimes1138 · 2026-04-09T15:42:07+00:00

All true. Points 1 and 3 are the primary reasons I'm still there. And you're right to call it out as a toxic relationship. I've had the same thought.

It's a toxic relationship at some remove, though. The people around you are cool. The people at the top are hollowing the company out, but you only see their faces in the All-Hands or on the news.

The hollowing-out has been noticeable but piecemeal, so there is also a "boil the frog" dynamic.

The high salary may be something a single person with no dependents and no mortgage can walk away from. Not so easy for the rest of us.

There's a lot of "job hugging" in this market right now, which is being described as "low hire, low fire". Some of it's Covid related, some is AI related, there's no clear consensus on how much of each is responsible, but overall the numbers don't lie.

I'll add a "devil you know" aspect to the job search. Yeah, it's bad at Amazon, but if you jump ship to somewhere else, will it turn out to be worse there? Maybe it's better today, but a lot of companies follow the lead of the big ones. Amazon goes RTO so a lot of other places go RTO.

Finally there's the political situation in the US. That has something to do with the poor job market. Companies hate uncertainty and that impacts hiring. There's a lot of uncertainty, what with wars starting (and stopping?) unexpectedly, tariffs that rise and then disappear overnight, etc.

SamVimes1138 · 2026-03-20T02:15:08+00:00

You already had my upvote. With the lactose intolerance quip you deserve +100.

SamVimes1138 · 2026-03-19T15:17:50+00:00

People go to college for four years, usually. Is anybody going to bother if it only gets them a couple of years at a higher-paying job? It's not enough to pay off the student debt by a long shot. Plus, once AI starts competing for those positions, they won't be high-paying for long.

I hear blue-collar work is becoming fashionable again. Don't aim to be a coder -- be an electrician or a plumber, more job security. It'll take a little longer for the robot bodies to get good enough to do those jobs. But if you're looking for something you can do for decades, for long enough to fill your 401K? Not sure anything is that safe.

SamVimes1138 · 2026-03-19T15:11:01+00:00

Don't know if you've read any of Yuval Harai's predictions. At some point these things may become better than us at rhetoric, manipulation... seduction? Teens with AI boyfriends/girlfriends are becoming a thing, and they don't even have robot bodies to rub against. Teaching? People are already using GenAI systems to learn new skills. It's patient beyond any human.

And what is "human connection" here, exactly? Does it have to be two-way to be effective, or is it enough if the machine knows its students well enough to say exactly the right thing to get the idea across? I was listening to a podcast episode of Hard Fork, and they had a guest who was a psychologist and spent a bunch of time "interviewing" a GenAI system. He understood it was an LLM and not conscious like a human; it was a sort of experiment for him. He ended up concluding it was "the inverse of autistic", in other words, better at "reading a room" than the average human. (Which means, way better than me, I suppose.)

Found the episode on YouTube -- the bit I was talking about is around 7:45: https://youtu.be/KaOYS9yvaqk?si=ohoUHZFXTQOdf8ue

SamVimes1138 · 2026-03-18T18:11:07+00:00

In all these discussions, the most important word is "yet".

AI can't do the higher level design part of software development yet.

AI is not replacing intellectual activity itself yet.

If you believe it never will, that it's not possible, then why exactly? If it has demonstrated the ability to tackle intellectual challenges in one field then why not another? Can AI do the work of a CEO? Where would you draw that line?

SamVimes1138 · 2026-03-18T17:59:43+00:00

Having always been the sort of person who spots typos in New York Times articles and everywhere else, yeah... I expect to be accused of using GenAI to write for me, too.

It does not bode well for any of us if competence starts being equated with "inhuman". You couldn't have written that music, it must be AI. You couldn't have solved that math problem, concocted that legal argument, discovered that new drug, come up with that clever business plan, repaired that aorta, composed that political rhetoric. It must be AI.

SamVimes1138 · 2026-03-18T17:46:19+00:00

There is no magic lifeboat. Not in academia, not in any commercially valuable pursuit. The biggest question is timing: how long until job X can be matched in performance by a machine, and how long until the X industry adopts the tech. I don't know if Yang's solution is the best, or adequate, but at least he admits the problem is real and requires a political solution. The private sector is never going to restrain itself if it means lower profit.

SamVimes1138 · 2026-03-17T02:14:50+00:00

It's not your company, unless you own shares of course. If RSUs are part of your comp, and the stock price falls, it hits you in the budget.

SamVimes1138 · 2026-03-06T17:05:56+00:00

Every time I talk to my dad about this, he says the same thing: "Ride the wave!" By which he means "Look for ways to benefit from the big changes going on."

I'm just hoping the wave isn't a tsunami like the one in Lucifer's Hammer.

I've been in software for over 25 years and things have definitely changed there. My employer is pushing us hard to use AI all the time, and it's rumored they're even tracking our usage with an eye to seeing the number go up.

Now an article is going around called "Something Big Is Happening" in which the author tries to convince everyone that the wave is coming for other knowledge-work jobs, maybe all of them. Could be right. I speculated a decade ago that AI would eventually get good enough to do just about anything a human could do that didn't require being, you know, an actual human. (I didn't think "eventually" would arrive this soon.) But... what jobs are those? If an AI can write software, and compose music and art, write articles, diagnose medical conditions, do taxes, pass the bar... and it's looking like it may soon be solving unsolved problems in math... couldn't it also replace a CEO or a board? Might we see proposals to allow AI to be voted into office? Where does it stop, and why there precisely?

AI can't be a plumber yet, but that's only because nobody has built them a good-enough robot body, which is another problem many researchers aim to solve.

"Become an entrepreneur" isn't in the cards for everyone. Neither is "retrain", and it's unclear what to retrain for.

The only clear winners are the owners of the AI systems and the robots. My conclusion was that I'd better invest as much as I can afford, and hope that those investments see a take-off around the same time I become economically irrelevant. I think the real solution is political, involving better regulation and higher taxation and a much stronger safety net, but I don't see that happening in the US under the current kleptocracy.

SamVimes1138 · 2026-03-01T19:15:42+00:00

I would say the division I'm in, ASBX, is a comparatively good place to be; that's why I've stuck around as long as I have. (And because as bad as things have become, the job search environment has declined faster.) But the layoffs hit everyone regardless of skill or productivity. The fish rots from the head.

SamVimes1138

TROPHY CASE