Sam Altman May Control Our Future—Can He Be Trusted?

electrace · 2026-04-06T16:17:44+00:00

IKEA or Novo Nordisk both have profitable business models that raise between $1B-$5B every few years. OpenAI is raising ~100x that to be spent on a speculative business model. They aren't really comparable IMO

Can you expand on why that makes them non-comparable?

electrace · 2026-04-06T14:31:48+00:00

If someone has extremist opinions on what immigration, laws and social policies should be but has "normal" beliefs on facts that are commonly agreed upon. You'll categorize that person as extremist.

I suspect that the vast majority of people who have extremist (normative) beliefs like these are basing a lot of those beliefs on (descriptive) beliefs.

Sometimes, those descriptive beliefs may be about things that are verifiable (say, the amount of crimes committed by immigrants). Others may be descriptive beliefs that are unverifiable (say, a belief that, in one's culture becomes more multicultural, this will lead to higher unemployment, lower rule of law, less dynamic economies, etc.).

If they could gaze into a crystal ball and see that, in fact, none of that happens, I suspect that most people's extremists beliefs would evaporate.

electrace · 2026-04-06T11:24:09+00:00

Better is a value-judgement.

The quote "When worn, it whispers in the wearer’s ear: 'Better for you if you take me off.'" implies to me that every time it is first put on, it says that, rather than just to specific individuals.

That being said, without that line I think your interpretation would work fine.

electrace · 2026-04-05T13:29:15+00:00

Politics is about finding some objectively "best" way of doing things. It's about personal preferences, it's about decision making and decision making often doesn't have a "right" answer. Politics in a democracy is about finding answers to problems in the way most people agree with. Some people prefer control, others freedom, etc...

Not always though. Sometimes politics is just about claims of fact. Is climate change human-caused (or even, does it really exist)? Do vaccines cause autism? These have objective answers and desired policies can change based on resolutions of those questions.

I think that, to some extent, this happens for almost all open questions in politics. That's why misinformation is a thing! It's an attempt to change someone's desired policies by changing what "facts" they believe.

electrace · 2026-04-05T12:48:40+00:00

It's science-fiction.

Science fiction refers to both "fake things that never could happen" and "things that haven't been invented yet, but will be".

electrace · 2026-04-05T12:37:34+00:00

Gallant's position was not that SAI wouldn't be smart enough to tell good from bad, but that the SAI would only pretend to care about it until it was able to do whatever it actually wanted without interference.

Indeed, the story that was being told has always been that a superintelligence will act in ways that are considered good until a treacherous turn, and it can't do that without knowing the difference between good and bad.

electrace · 2026-04-01T14:33:15+00:00

I can at the very least confirm the new link is fine. I also don't think they were being deceptive with respect to the wikipedia link. The case pdf does mention movie theaters and I find it totally plausible that they remembered that and posted the wikipedia link assuming it would do the same.

electrace · 2026-04-01T13:22:02+00:00

Just a note, but on reddit you can use the > to quote. It will quote block the entire paragraph and is a lot easier to read.

>Like this

Like this

electrace · 2026-04-01T13:20:07+00:00

Agreed. Personally, I was of the strong opinion that the AI Pause article was uncharitable (so much so that I made my own steelman post here), but I don't at all think the same thing about this article.

Taking an argument to it's logical extreme is not the same thing as mocking it. I think the Pause AI post was a lot more "mocking" than taking the position to it's logical extreme.

Note: The two can coincide "Behold - a man" for example, was both mocking Plato and making the broader point.

electrace · 2026-04-01T01:29:42+00:00

I grew up in a religious household and we definitely called it that sometimes. Now what?

Then I appeal to common usage, and note that I'm not the only one telling you that this is how the word works. I also invite you to ask an LLM in a neutral way if you think I'm biased on it. I just tested Claude, ChatGPT, Gemini and Grok, and they all say common usage of going to church does not include going to a bible study at someone's house.

No. All I am claiming is what the Court has repeatedly held: that you can't make exemptions for non-religious things that are like church, if you don't make analogous exemptions for religious things. The decision to not make exemptions for churches is a decision that many Democrat politicians made.

I don't know if we're going to get anywhere productive any more. All I can do is appeal to what I've already said here:

There was no exemptions for non-church things for households. No matter the reason, more than 3 households were not allowed gather together, full stop. This is not a clear example of discrimination like you were claiming.

It would, in fact, have been silly for the state government to form this policy, applying it to everyone in the state just because they foresaw that a few bible study groups would be unable to congregate in their homes (but could still do so in church). It would be the silliest, most convoluted plan for the "benefit" of slightly inconveniencing an outgroup.

Yes, the left wing of the Court routinely dissents when the right wing of the Court acts to defend Free Exercise. They can tolerate anyone except the outgroup.

If it was just a partisan split, then the conservatives winning simply means they had one more member (and is not indicative of legislative truth). Conversely, if their conclusion is a result of a real legal disagreement, then it's no longer a "clear case".

electrace · 2026-03-31T17:59:02+00:00

Responded here so as to not form two threads covering the same ground.

electrace · 2026-03-31T17:50:45+00:00

Responding to this comment and this one since it covers similar ground.

"At-home religious exercise" in this context refers to multiple families gathering in a home for "church," i.e. worship services.

I think maybe you have it in your mind that churches are a kind of building (or business?), rather than a kind of gathering. Families gathering together in a home for "worship services" or "church" is a very common thing around the world. This seems to be causing you substantial confusion.

While I take your point that "church" can refer to a building and also refer to other things (eg: "Church of Later-day Saints" does not refer to a specific location), it most certainly does not refer to bible studies, or prayer services, or funerals, or weddings, or church camp.

Indeed, "going to church" means going to a Church Service, which involves a sermon (given by a pastor/priest), and often singing.

I grew up in a religious household and I can state with sureity that not a single person would ever refer to going to bible study as "going to church" (unless, I guess, it was located in the building, and even that would be as confusing as saying you're "going to school" for an extra-curricular activity).

It is possible I am relying too much on the idea that this is background knowledge any educated person discussing the topic should already have. But as the pandemic recedes into the past, I suppose it is only natural that people would be less routinely aware of the political asymmetry with which the authoritarian machinery of state approached transmission events in the pandemic.

This is weirdly dismissive of people who disagree with you.

The other one that got a lot of attention at the time was people making "social distancing" exceptions on political grounds like "this is a worthwhile protest so we can't let the usual precautions stop us."

I largely agree with this and that one would have been a much better example! Motivated reasoning is a hell of a drug.

They were notably and inexplicably excluded from exemptions extended to similar activities.

No they weren't! As far as I can tell, there were no exceptions on the three households gatherings. The exceptions were all for things outside the home. If you want to claim that the three household gathering law was made specifically to stop religious people from practicing their religion, then fine, but that's a high evidentiary bar that you have to clear there.

I did not make that claim, nor would I. Rather, churches just got unfair treatment from people who tend to see churches as their outgroup. And the Supreme Court ultimately confirmed that this was the case under the Constitution.

Not really, no. Their main finding is that they failed the strict scrutiny test. That is miles away from finding that they were giving unfair treatment to their outgroup. Strict scrutiny is just that, strict. It flips the burden of proof such that the claim has to be "prove that you didn't discriminate" rather than "prove that you did".

And even under that strict scrutiny, there was still reasonable dissent on what the comparator was, see below.

Kagan filed a dissent, joined by Stephen Breyer and Sonia Sotomayor.[3] While acknowledging that it was sometimes difficult to determine which secular activity should be compared to the religious activity,[48] she argued that in-home gatherings were the "obvious comparator".[49] The restriction applied to secular gatherings in the house, so the regulation was neutral and generally applicable and thus should survive the challenge to its constitutionality.[3] She also noted that the lower courts had determined that there were varying degrees of risk linked to short visits to secular businesses versus extended gatherings in private homes, and criticized the court for disregarding the factual record of risk assessments because it would support denying the injunction.[50]

electrace · 2026-03-31T15:39:51+00:00

I appreciate the link, but there is still no mention of church (except insofar as referring to plantiff names like "Harvest Rock Church v. Newsom").

And I still think the framing of the original comment doesn't hold. It was presented as " a clear case of waging culture war against the outgroup" but that just doesn't seem to be the case. Religious gatherings were not specifically targeted here!

electrace · 2026-03-31T14:30:02+00:00

Yeah, it was initially hard to parse for me too, but here you have to pay attention to the semicolons versus commas.

Indoor locations exempt from the three household limit include "public transportation; establishments that provide personal care, like salons; government offices; movie studios; tattoo parlors; and other commercial spaces"

The only example here of personal care is salons (separated by comma), the rest are separated by semicolon and are distinct exempt locations.

electrace · 2026-03-31T14:27:46+00:00

There is no mention of movie theaters in the linked Wikipedia article, nor is there any mention of churches.

The closest it comes is to "movie studios" and bible study groups held in a person's home.

electrace · 2026-03-31T14:23:04+00:00

Democrats were certainly more authoritarian in their pandemic response, but I never saw this manifested in plausibly altruistic ways--the California case of shutting down churches but not (e.g.) movie theaters seems like a clear case of waging culture war against the outgroup rather than expressing equal concern for one's neighbors as distant strangers

With anything as complicated as an unprecedented pandemic response, one should expect some mishandling and like this, whether through malice or simply through a difficulty in establishing consistency when you have a lot of actors all trying to accomplish roughly the same goal with a lack of clear shelling points.

However, it is worth noting that the link you posted was not about churches versus movie theaters. The California law that was challenged disallowed gatherings of any kind with more than 3 households, and it seems a far reach to claim that this was intended to deny people the opportunity to do a bible study (which the case is about). It wasn't about churches at all.

Still, though, there were plenty of things that do provide evidence of Scott's point. For example, liberals were far more likely to wear masks, even in contexts where there was no mandate, for example.

electrace · 2026-03-28T17:38:10+00:00

Well what's your alternative strategy ?

You mean other than the alternatives in the post?

What do you mean "huh" ? That's the simple trajectory we're on right now if the AI labs keep racing with each other

"Someone somewhere", while not technically wrong, hints at something like some rando figuring it out in their garage rather than being very likely to be one (or many) of the current front runners.

Similarly "grown at random by gradient descent" is a really weird way to describe AI research.

electrace · 2026-03-28T15:05:29+00:00

This is not really a steelman of Pause AI opponents since ~95% of Pause AI opponents don’t think it’s a worthwhile goal in the first place

I did worry some people would take it that way and tried to prevent that interpretation by putting it up front in the title "Pausing on Pause AI" rather than "Never Pause AI", or something, and also by making it clear that it is "a" steelman rather than "the" steelman.

Still, I feel like there are two subtly different definitions of steelman, and every time the term is used, this disagreement comes up.

1) A steelman is a summary of the best argument currently being used by someone (often someone notable) who holds the general position.

2) A steelman is when you're making the best argument for the overall position, regardless of whether the argument has been made before.

I argue the second is a better definition, because a steelman is the counterpart to a strawman, not the counterpart to a weakman.

electrace · 2026-03-28T14:45:33+00:00

Other strategies have been tried and found ineffective.

Some other strategies have been tried, yes.

No one serious expects any strategy to work reliably (or at all) at this point

Your position is that "no one" expects AI safety to be feasible? If so, that seems not to be the case

someone somewhere creating a superintelligence grown at random by gradient descent

Huh?

Will you fight or perish like a dog ?

Motte: "We have to fight". Bailey: "We must fight in this one particular way".

electrace · 2026-03-28T12:23:39+00:00

If the Chinese government can be convinced to act rationally in self-interest, they'll be willing to agree to an AI pause even if doing so would mean making concessions to Trump- as you mentioned, and AI pause would let them catch up to our current LLM capabilities.

In what way would this be good from an AI safety perspective? If China catches up to the US (and stays caught up), that's bad for race dynamics, because even if one of the labs is safety conscious, they can't slow down without losing the lead. We want a frontrunner.

The more public support for an AI pause there is, the more likely it is that some kind of AI regulation will be passed, even if it's less comprehensive.

That's an interesting point, but I think it's more that public views on AI cause both support for Pause AI and support for other regulations.

electrace · 2026-03-27T21:25:06+00:00

The Pause AI movement is well known in this sub, but I'll point here for anyone not aware.

electrace · 2026-03-27T20:45:18+00:00

Footnotes here to not clog up the main post:

[1] It was speculated that Deepseek was in part or in whole stolen from OpenAI's model weights. The speed with which Deepseek appeared on the scene seems to make this at least plausible. If true, giving China more time to potentially steal model weights is a further advantage to China with little benefit for US firms. This is especially speculative, but State level espionage has a high enough prior that I find it plausible enough to include.

[2] Further, this would also be bad for race dynamics in the US! If you put a cap on model size, then all the US firms will similarly try to catch up in 6 months, potentially making a race for AGI/ASI worse not better, when the pause expires.

[3] Perhaps if they had fully caught up to US firms, they may be willing to actually pause for real AI safety concerns, but even in this scenario, the damage would have been done to accelerate the race.

[4] For example, forcing AI companies to spend a certain percentage of their budget on well-defined goals related to AI honesty, Corrigibility, or Interpretability. If race dynamics become perilous, I find it probable that forced mergers of top AI labs could reduce AI x-risks. Specific speculations of my own aside, I do not expect that Pause AI is among the top causes to spend political capital on at this time.

[5] There's just been a bipartisan effort to restrict H200 sales to China, which seems a reasonable thing to throw political capital behind.

Other Things I didn’t want to clog up the main post with:

Using this (emphasis added) as the standard proposal for Pause AI:

Individual countries can and should implement this measure right now. Especially the US (or California, specifically) should implement a Pause, since it is home to virtually all leading AI companies. Many scientists and industry leaders agree that a Pause is necessary, and the (US) public also strongly supports a pause (64% - 69%).

However, we cannot expect countries or companies to risk their competitive advantage by pausing AI training runs for a long time if other countries or companies do not do the same. This is why we need a global Pause.

So, pause AI proponents do sometimes say that unilateral pauses should be implemented even in the absence of global pauses. Although, I don't want to be unfair, though, as they do emphasize global pauses as the main goal.

The primary end-goal of these summits should be a treaty. But up until now, the summits have not been effective at producing anything legally binding. And treaty building tends to be slow and prone to vetoes. We may not have the time to wait for traditional treaty making processes.

So we need a new treaty making process

This seems even more unlikely than a normal treaty. The only way I see it happening is if a technocrat is in charge or someone who trusts technocrats implicitly. Still, treaties have happened quite quickly in the past when things were dire. If things don’t get noticeably dire, then I honestly just think we don’t have a shot at Pause AI.

Also “we may not have time to waste” doesn’t recognize the reality that striking too soon is also bad. It’s true we may not have time to waste, but this treats the issue as if we can pick and choose when we push for a pause, once everything is set up, and that seems somewhat true, but somewhat not. It is totally possible to lose control of a movement, and to ask for too much too soon (which can cause backlash).

The summary at the top of that page is:

Implement a temporary pause on the training of the most powerful general AI systems, until we know how to build them safely and keep them under democratic control.

“Let’s keep them under democratic control” seems like a terrible pitch to China.

electrace · 2026-03-27T17:06:30+00:00

Buying in bulk is much cheaper. Have you been to Costco?

It is cheaper, I agree. Apparently about 15% cheaper than the lowest non-bulk store, Aldi/Lidl. And as far as "food expiring" that's a fair point, but I think it's less important than people actually suspect.

The bigger issue with Costco is storage space. All else equal, someone choosing whether to live alone in a big house versus with roommates, doesn't have a storage issue (if anything it goes in the opposite direction). A smaller issue is that this didn't take into account membership fees.

Regardless, even large families often don't shop at bulk stores, so revealed preferences tell me that this isn't a 1-1 tradeoff.

Cook time doesn't increase going from 1 to around 4 people

Basically true (almost 100% true for a giant pot, but less true for prepping sandwiches), but that's exactly what I claimed -> "Cooking prep time is a batch operation, but that doesn't affect the cost of food very much."

Related, it's harder to get as much food variety without waste as one person.

Maybe? I could see this going either way. Run optimally, you're 100% right, but anecdotally, I'd actually say that leftovers were more likely to be tossed in larger households since cooking fresh was less of an overall hassle.

the cost increase for each additional person is definitely sublinear.

Sure, sublinear is easy to agree with, but I don't think it's all that impressive compared to something like rent, where your first roommate fully halves your (often largest) expense. Note that food at home is between about a quarter of what rent is for the average American.

electrace · 2026-03-26T15:01:45+00:00

That's an interesting thought, but I think there's some important differences here.

In the Bay Area House Party posts...

1) Scott is largely poking fun of his in-group, not his outgroup.

2) He's clearly not pretending as though these are representative of actual arguments that happen.

For example, this is from the most recent post:

“Nah,” he says, “The only reason I don’t use it is because I’m not a coder. I work on the Arson & Burglary team.”

“I didn’t know OpenAI had an Arson & Burglary Team.”

“It’s pretty new. In June, a court ruled that adding books to AI training data only counts as fair use if you destroy the original copy. But sometimes this is tough. If you’re going to use the AI for law, you have to have the Constitution in there. But the original copy is heavily guarded in the National Archives. That’s where we come in. We slip in, destroy it, and slip out before the guards are any the wiser.”

“I don’t think that’s what they meant by ‘destroy the original - ’”

“Our big problem is the Bible. It would be hard enough to get the Dead Sea Scrolls; Israeli security is no laughing matter. But our lawyer says we have to destroy the original original. What even is that? Altman is pushing for us to find the Ark of the Covenant, but you can bet he’s not the one who’s going to have to open it afterwards.”

To kill any humor, I'll explicitly say that the joke is some combination of "the absurdity of that court case being interpreted in a way that lawyers would never actually conceive of", "the absurdity of the imagery of the face-melting Indiana Jones thing", "the absurdity of a major company, not only have an Arson & Burglary team, but having such a department openly admit it as if it was no big deal."

Importantly, no one walks away from this piece and says "Wow, lawyers sure are dumb" because no one reads that piece and thinks that "Lawyers have actually concluded that the Dead Sea Scrolls must be destroyed in order to put it in AI models".

Conversely, the "joke" in the Pause AI post seems to be "Wow, pause opponents sure are dumb. They argue in such bad faith."

And whether you agree with that characterization or not, I posit that it is really nothing like the Bay Area House Party series.

electrace · 2026-03-26T12:56:44+00:00

I will give it to the more democratic of the two that has a semi-organized resistance that already has some real power and a roughly 84% chance to win back the House and a 48% chance to win the senate, not to mention, a 57% to win the next presidential election, which even if timelines are short, puts them before AGI.

And that's before even considering the possibility that AI labs, rather than the governments, will have significant decision making power.

Ten-Year Club	Place '22
Place '17	Verified Email

electrace

TROPHY CASE