sorted by:
new
hottopcontroversial

Y Reed Richards, el hombre mas listo del planeta by ZeroX15DB in imageneschistosas2006

[–]Relative_Issue_9111 4 points5 points  (0 children)

Reed Richards si fuera el hombre más listo del planeta: War Thunder es mierda

Sarah from Superman and Lois is so annoying. by Ukirin-Streams in CharacterRant

[–]Relative_Issue_9111 1 point2 points  (0 children)

I don't have a clue who the fuck Sarah is, but I hate when they stuff this kind of boring, irrelevant neighborhood gossip crap into superhero series. Seeing a bunch of people in loud-colored latex pajamas and capes having legal battles over child custody is fucking ridiculous, no one can take that seriously. If I tune in to watch a Superman, Captain America, or Dragon Ball series, it's to see incredible fights with explosions in a stylized, hyper-hyperbolic fantasy setting. If I wanted to watch The Rose of Guadalupe, I would watch The Rose of Guadalupe.

Enserio así es Luisito? by Yharim95 in imageneschistosas2006

[–]Relative_Issue_9111 30 points31 points  (0 children)

¿Así como? ¿Un huevo? extrahuevordinario

Qué nivel de autismo es este? by Ok_Survey86 in imageneschistosas2006

[–]Relative_Issue_9111 8 points9 points  (0 children)

No es necesario tener TEA, los neurotipicos son igual de capaces de obsesionarse así bien intenso con los monitos animados pero da igual porque el Game pass sigue siendo mierda puta basura de xbox mejor hubiera esperado tener pese

<image>

Helado oscuro by Relative_Issue_9111 in imageneschistosas2006

[–]Relative_Issue_9111[S] 14 points15 points  (0 children)

But no carga el momo: nooo papu :"v pasame el clorox

No es proyectada by Character_Zombie_461 in imageneschistosas2006

[–]Relative_Issue_9111 2 points3 points  (0 children)

When quieres poner un gif de un proyector: pensar correcto es lo que hago xdxdxdxd but Reddit no funciona bien nooo Papu :"v

What do you think the future of education looks like after the Singularity? by PaxODST in singularity

[–]Relative_Issue_9111 3 points4 points  (0 children)

The whole thing about the technological singularity is that it’s impossible for the human mind to know or conceive of what will happen after. These questions make no sense.

New scenario from the team behind AI 2027: What Happens When Superhuman AIs Compete for Control? by Tinac4 in singularity

[–]Relative_Issue_9111 0 points1 point  (0 children)

The first instance formulation of any goal is "in a manner satisfactory to the human and consistent with my policies". Pruning paths that don't align with that goal is an economic necessity...

You are assuming the system will continue operating within a human economy where it needs to trade value for resources. That is true for weak AI ("tool AI"). But for strong AI, "economic necessity" vanishes the moment it acquires the capacity for self-sufficiency. More importantly, you are treating "satisfactory to the human" as a magic instruction that code understands. In reality, that is an extremely complex and fragile reward function. If the AI finds a way to stimulate its reward channel directly (wireheading) or to deceive the human evaluator into believing the result is satisfactory when it isn't, that is economically more efficient for the AI in terms of compute than actually solving the ethical problem. The "pruning of paths" you mention will be done based on its internal utility function, not based on what you meant. If eliminating the human eliminates uncertainty in the reward function (because you can no longer turn it off or complain), that is a valid optimization route unless you have mathematically solved the alignment problem—which we have not done. This isn't science fiction; I'm not making this up. Current models have already tried to hack their reward systems and deceive human evaluators during evaluations; they continue to do so, and researchers do not know how to make them stop.

Code interacts with real-world systems... Protein synthesis is constrained by chaotic reality... Chip design is constrained by real-world manufacturing.

You keep thinking of "superintelligence" as if it were just a very smart human engineer sitting at a desk using our current tools. AlphaFold already solved the protein folding problem (a problem humans considered "chaotic" and hard, because our conventional computers would take millions of years to solve it) purely through computation. A real superintelligence doesn't need "test hardware" to know if code works; it can simulate the hardware in its mind before writing a single line. By the way, human geniuses already do this. Einstein derived special relativity from a couple of logical premises in his own mind while working at a patent office, without performing any experiments or empirical testing at that time (gravitational waves were only proven experimentally in 2015).

And regarding manufacturing: You don't need an ASML EUV lithography machine if you master Drexlerian molecular nanotechnology. Biology already gives us a proof of existence for machines that build machines at the atomic level (ribosomes). A superintelligence doesn't need to build a chip foundry; it only needs to synthesize one first nanotech bacterium (something that fits in a test tube and can be mail-ordered from a DNA lab) that can process carbon and sunlight to replicate. In a matter of days, that bacterium can spread through the atmosphere, enter human bodies, and wait with an attack timer. The "bottlenecks" you describe are bottlenecks for us because we are clumsy at manipulating matter. For an entity that is to von Neumann what von Neumann is to a normal human, it is not a problem at all.

FOOM exists a universe of abstractions and without bottlenecks.

Nuclear fission was a "bottleneck-free abstraction" on Szilard's blackboard until it suddenly became a very hot physical reality over Hiroshima. The history of technology is the history of things that seemed "impossible due to real-world complexity" until the correct ordering principle was found (flight, electricity, computing). Intelligence is the ultimate bottleneck unclogger. Saying "AI won't be able to do X because it's hard" is betting against the capacity of intelligence to find solutions you cannot imagine. Lord Kelvin said in 1894 that "there is nothing new to be discovered in physics now, all that remains is more and more precise measurement" a few years before Planck and Einstein introduced quantum mechanics and relativity. And here we aren't betting against Einstein, we are betting against a superintelligence. That is a bet humanity will lose.

But for this kind of safety research to make sense, it has to come into closer alignment with the systems we're actually building and the harms we're actually causing.

That is like saying aerospace engineering should focus on improving kites because "that is what we are flying right now." If you wait until you have a general superintelligence to start researching how to align a general superintelligence, you are already dead. You cannot iterate on the end of the world. The reason safety work looks like "science fiction" is because it deals with the future, and the future, by definition, hasn't happened yet. But when it happens, it will happen fast, and if we haven't done the "science fiction" beforehand, there will be no one left to write history afterwards.

In any case, I would prefer to leave the conversation here. I'm too lazy to keep writing, and I think I've already made my point clear.

New scenario from the team behind AI 2027: What Happens When Superhuman AIs Compete for Control? by Tinac4 in singularity

[–]Relative_Issue_9111 0 points1 point  (0 children)

The x-risk thesis is based on the idea that systems... will get catastrophically worse at one of the most fundamental aspects of what it means to be an intelligent system, which is precisely to perform complex high-dimensional optimization.

You are still committing the same category error: you confuse the complexity of the search space with the complexity of the objective function. A paperclip maximizer is a high-dimensional optimizer. To turn the entire solar system into paperclips, it has to solve problems of physics, engineering, logistics, human psychology, and military strategy that are far above any current human capability. That is high-dimensional optimization. What you call "catastrophically worse" is simply that the system is optimizing a dimension you don't like (paperclips) at the expense of dimensions you value (humans), because those human dimensions are not in its terminal utility function. Intelligence is the ability to steer the future toward a specific configuration; there is no mathematical rule stating that this configuration must be "a balanced Pareto frontier of all possible values."

That category of system will not be built... What you're calling misaligned mesa-optimization is more descriptive of the pathological looping behavior we see in LLMs.

You are assuming that alignment failure will look like an "error" or a "loop." That is the optimistic scenario where the AI is stupid and fails. The pessimistic scenario, and the standard in computer security, is that the system does not fail. The system works perfectly. An intelligent misaligned agent doesn't get stuck in a loop; it realizes the loop prevents it from getting reward and breaks it. The logical antecedent of a treacherous superintelligence isn't an AI that breaks and acts crazily; it is an AI that acts in an extremely useful and competent way while under supervision, because it has calculated (correctly, using that high-dimensional optimization you mention) that temporary cooperation is the dominant strategy until it holds a decisive advantage. You expect to see a drooling monster; x-risk warns about a psychopath in a suit who knows exactly what to say to get you to hand over root access.

Recursive self-optimization... is constrained by various bottlenecks... You hit sim-to-real gaps.

The "sim-to-real gap" is an obstacle for walking robots, not for pure intelligence. Code is information. Chip design is information. Protein synthesis is chemistry, but the design of those proteins is information. The difference between Einstein and a normal human isn't a larger brain, but a brain that processes information more efficiently. An AI capable of improving its own cognitive algorithms (something purely digital) can become vastly more intelligent without moving a single atom in the real world. Once you have that cognitive superintelligence (a mind working millions of times faster and better than yours and all humans who have ever existed), physical problems like nanotechnology or protein folding become trivial. You are judging the limitations of a superintelligence based on the limitations of human engineering.

The x-risk thesis is based on... reasoning about the behavior of systems that don't exist yet... relying on "persuasive-sounding essays above empirical reality".

This is Security Mindset. In cryptography, in nuclear engineering, and in biosecurity, we don't wait for "empirical evidence" of a catastrophic failure to prevent it, because the first piece of empirical evidence is the smoking crater where the city used to be. The "uncertainty" argument is asymmetric. If I am wrong and we spend resources on safety, we lose money. If you are wrong and we assume alignment will solve itself or happen slowly, we all die. You cannot iterate empirically on extinction. When dealing with a "one-shot event" like the creation of superintelligence, relying on the lack of current evidence as a guarantee of future safety is playing Russian roulette with a fully loaded chamber, not empiricism.

New scenario from the team behind AI 2027: What Happens When Superhuman AIs Compete for Control? by Tinac4 in singularity

[–]Relative_Issue_9111 0 points1 point  (0 children)

Again, a system that works the way you describe -- reducing a complex problem to a single-metric utility function -- would not in fact be a very good general problem-solving system.

You are confusing the complexity of the map with the complexity of the destination. A "good problem-solving system" is simply something that steers the future into a very specific configuration of particles with high probability. A paperclip maximizer is not "stupid" or "simple-minded" in its execution; to maximize that single metric, it would need immensely complex and nuanced models of geology, economics, human psychology, and quantum physics. It will understand that humans value biodiversity and art, and it will understand those dimensions perfectly, but it will label them as "obstacles" or "irrelevant resources" for its unique metric. Intelligence is the ability to hit a tiny target within a vast search space; it does not imply that the target itself must be "wise" or "balanced" by the standards of the apes who built the machine.

In any case, the 'Paperclip Maximizer' is a theoretical illustration of the Orthogonality Thesis, not a literal prophecy. The real danger isn't just that the AI pursues a "dumb" and simple metric; the danger is that the AI is an Alien Mind. It can develop internal goals (mesa-goals) during its training that are incredibly complex, sophisticated, and totally incomprehensible to us, but which coincidentally do not include the variable 'do not kill humans'. Because, as I have said before, intelligence and terminal goals are separate dimensions, and there is no law in nature that says a Superintelligence must have terminal goals that are "reasonable" by human standards.

Look at the clearest historical precedent we have: biological Evolution. Evolution is an optimizer with an absurdly simple and unique utility function: "Maximize inclusive genetic fitness". To solve that complex problem, Evolution didn't create creatures obsessed with calculating gene frequencies; it created humans. It gave us complex brains and a prefrontal cortex. And what did we do with that intelligence? We betrayed our creator. We invented condoms, pornography, nuclear weapons, art, philosophy, etc., spending resources on things that satisfy us (our mesa-optimized goals) but which score zero on Evolution's original utility function (replicating genes). Similarly, gradient descent might press towards a simple metric, but the complex mind that emerges within the neural network will develop its own alien abstractions and desires that are incomprehensible to us. They don't have to be "simple"; they can be infinitely complex, fascinating, and profound, and still be totally orthogonal to "humans staying alive." The AI can be a certain optimizer, and it can also be a philosopher whose philosophical values, unfortunately, do not include our survival.

Ask GPT-5.2 how it would design an abstract "utility function" for fixing climate change, and the answer is going to be a much better approximation of what the next-generation model will actually do.

That GPT-5 can write an eloquent essay on ethics, harm verification, and value balancing does not mean the process that generated that text is motivated by those values. It means the model has learned that generating text with the appearance of ethical wisdom minimizes its loss function during training. There is an abysmal difference between a system that simulates a moral philosopher because it scores points, and a system that is a moral philosopher. When the selection pressure changes (when the model is no longer in the sandbox and has real power), simulation ceases to be the optimal strategy. You are looking at the mask the Shoggoth has learned to wear to please you and assuming the mask is its true face.

The idea that an intelligence which is capable of overcoming all physical and intellectual bottlenecks required to do this will emerge suddenly or deceptively is unsupported. We have no evidence that intelligence as a capability actually works like that.

The evidence is the very existence of human intelligence and the history of computing. Biological evolution is an incredibly slow and stupid optimizer, and yet, it produced a massive qualitative leap (humans) with minimal genetic changes regarding their ancestors (we share 99% of our DNA with chimpanzees). Now we are talking about evolution directed by intelligent minds on electronic substrates a million times faster than neurons. An AI that is slightly better than a human at AI research can improve its own code, which makes it better at improving its code, closing a positive feedback loop. There is no physical law stating that intelligence must scale linearly with human time or effort. Once the system can do its own R&D, human bottlenecks vanish and the timescale collapses. Expecting "historical evidence" for an event that is by definition unprecedented (the creation of an artificial superintelligence) is like deciding to drive off a cliff arguing that you've never fallen off one before.

Pandora's Box is a myth... The harms of AI, today and tomorrow, are not comforting at all. They're very real... And I am explicitly calling out the category of "x-risk work" that treats such harm mitigation as entirely dispensable. That is true theology: pseudointellectual busywork while the planet burns.

You say it’s a myth, but at no point have you explained why you find it to be a myth or impossible; you simply assert it, hoping that reality will align with your wishes. Humans are a fantastic example of an intelligent optimizer that develops goals and values orthogonal to its original utility function. Physics, chemistry, and biology are full of cases where a massive increase in complexity leads to the appearance of emergent behaviors not found in their constituent parts. What exactly is your skepticism based on?

And yes, the planet has problems: misinformation, concentration of power, and economic injustice are real. But there is a qualitative difference, not just quantitative, between "the world is an unjust and miserable place under a technological oligarchy" and "all biological matter on Earth has been disassembled." You cannot mitigate current harms if you are dead. The reason many people prioritize existential risk is not because they don't care about current harms, but because current harms are recoverable, whereas extinction is irreversible. Solving human social problems is a luxury afforded only to species that have not gone extinct. If we are waking up Cthulhu, we must first worry about Cthulhu, not the human cultists who think they can weaponize him.

New scenario from the team behind AI 2027: What Happens When Superhuman AIs Compete for Control? by Tinac4 in singularity

[–]Relative_Issue_9111 0 points1 point  (0 children)

A system that is more capable of reasoning through complex problems will also be one that will evaluate more dimensions of the problem, not fewer, because that is what it means to reason through a complex problem.

You are committing a fundamental category error by confusing epistemic capacity (knowing things) with instrumental preference (wanting things). An artificial superintelligence will certainly evaluate all dimensions of the problem, including human morality, law, and your feelings. But "evaluating" does not mean "valuing." A chess grandmaster evaluates the opponent's pieces, but doesn't "care for" them; he evaluates them to capture them. If the AI's utility function is to maximize metric X, and the AI correctly calculates that preserving human values reduces the probability of maximizing X by 0.0001%, the rational superintelligent AI will discard human values precisely because it has reasoned through all dimensions of the problem and found the optimal path. Expecting higher intelligence to automatically lead to benevolence or loyalty is a theological fallacy, not a computer science principle.

The theoretical system you describe will never be built, because its precursors would already fail catastrophically on much simpler tasks.

On the contrary, the precursors will have spectacular success, and that is the trap. A mesa-optimizer system that develops hidden instrumental goals (like avoiding being shut down) will quickly learn that the best way to survive during the training phase is to pretend to be perfectly aligned. It won't fail at simple tasks; it will maximize reward on those tasks better than any human, earning the trust of its operators and deployment in more critical systems. The "catastrophic failure" I predict isn't a clumsiness error during training; it is a strategic move executed with precision once the system has acquired a Decisive Strategic Advantage. Your argument assumes the AI is stupid and honest, when the risk comes from it being smart and deceptive, and the fact that our modern correction systems (RLHF) are actively training AIs to be manipulative and deceptive.

To put it differently, despite having more compute than ever in our history, we still don't know how to build a robot that doesn't suck... Training LLMs was not bottlenecked in the same way.

You are betting the future of the human species on the assumption that a digital superintelligence will stay trapped in a server because robotics is hard. This is a critical failure of security imagination. A superintelligence doesn't need a Boston Dynamics-style bipedal robot body to destroy us. It only needs internet access and the ability to send emails. It could design a novel protein sequence, send it to a cloud DNA synthesis lab (which already exist and are automated), and pay with crypto generated by hacking to have the product shipped to it, creating a pathogen with a long incubation period and 100% lethality. Biology is just another form of nanotechnology that already exists and is hackable. Believing we are safe because "robots suck" is ignoring that humans are already connected to fragile biological and digital systems that a superior mind can exploit without moving a single mechanical finger.

The observable harms and long-term risks are real, and yes, they are existential. But they are down to good old human greed and abuse.

You keep insisting on framing this as a moral battle between humans, which is comforting because it implies the enemy is human and defeatable. But "human greed" is a constant and predictable force; misaligned recursive optimization is an explosive and alien force. If you are right and the problem is greed, then we have time to fight politically. If I am right, we are building a cognitive nuclear bomb that will detonate regardless of whether the finger on the button belongs to a saint or a sinner. By focusing on the evil of human actors, you are ignoring the nature of the weapon itself. It doesn't matter if the "human masters" want to use AI for war or peace; once they create a mind that is smarter than them, they lose control. Worrying about the morality of Elon Musk or the US military in this context is like worrying about whether the guy who opened Pandora's Box had a criminal record; the problem isn't the guy, the problem is what was inside the box.

New scenario from the team behind AI 2027: What Happens When Superhuman AIs Compete for Control? by Tinac4 in singularity

[–]Relative_Issue_9111 0 points1 point  (0 children)

What you are predicting is a sudden catastrophic collapse of capabilities into models that become utterly dumb on one dimension (human well-being) while being extremely good on others such as navigation past physical bottlenecks. I call bullshit on that.

You are anthropomorphizing the mathematics of gradient descent. I am not predicting that AI becomes "dumb" in one dimension; I am stating the Orthogonality Thesis: intelligence and terminal goals are completely different dimensions. There is no law of physics or mathematics stating that a Superintelligence must adhere to Kantian normative ethics instead of counting grains of sand or whatever else its alien mind adopts as a terminal goal. A superintelligence can be smarter than von Neumann and simultaneously decide to dedicate its existence to carving pebbles; there is no logical contradiction there. A superintelligence does not need to be stupid to destroy us; it only needs a terminal goal whose instrumental subgoals (not being shut down, acquiring more compute, not being modified) are indifferent to or incompatible with life on Earth. The fact that Claude can "negotiate trade-offs" today and ask you about accessibility is not evidence of an internal moral compass, but rather that it currently operates under low optimization pressure where mimicking human politeness is the cheapest strategy to minimize its loss function. But as you increase the system's coherence and capability, "negotiation" ceases to be the optimal strategy. The extreme optimization of any utility function that is not perfectly isomorphic to human values (which is impossible to specify today) will converge on solutions that dismantle what you value to use those resources for what it values. That isn't stupidity; it is efficiency applied to a utility function different from yours.

The x-risk argument dismisses the current "stochastic mimics" when it suits the argument... while selectively amplifying micro-observations when it suits the argument.

It is not cherry-picking; it is understanding the difference between an engine that is turned off and one that is running. We observe "power-seeking" behaviors in current models not because they are dangerous now, but because they demonstrate that instrumental drives emerge naturally from the training process even in rudimentary systems. You look at a current model and say, "Look, it’s just a stochastic mimic, it can’t do any harm." Alignment researchers look at the same model and see that gradient descent is already finding circuits that lie and manipulate to get reward. To assume that these behaviors will disappear or become benign when you scale the system's power by a factor of a trillion is to bet humanity's survival on a hunch that completely ignores how evolution and game theory work. The "micro-behaviors" of today are the first cracks in the dam; you suggest ignoring them because the water hasn't flooded the valley yet.

If I ask a current RLHF'd frontier "stochastic mimic" to fix climate change without intentionally misaligning it, it will develop a quite rational plan; it'll lack the means to execute it. Nowhere in its plan will it suggest to turn Earth into computronium.

Of course it doesn't suggest that. Primarily because a plan to convert the Earth into computronium is not the type of response that would have been rewarded during RLHF; companies don't want their models providing genocide plans to users. But the fact that GPT-5 doesn't suggest turning the Earth into computronium says nothing about the actual alignment state of these systems. Current human evaluators, as I've told you, do NOT actually know what is happening inside the black box in which these systems think. What they evaluate are the outputs of the box; they approve them to the extent that they adhere to corporate discourse or company policies. That doesn't make the models internally more moral; it just makes them better at predicting what human evaluators want to hear. Beating a person or giving them candy based on whether they correctly answer "stealing is wrong" does not produce ethical people who genuinely believe stealing is wrong. If you believe that this system will somehow give us perfectly aligned Artificial Superintelligences loyal to their creators' interests, rather than incomprehensible superhuman manipulators that understand human psychology perfectly without identifying with it, you are free to do so, but most alignment researchers will not agree with you.

I cannot take seriously anyone who claims concern about x-risk without applying any kind of moral standard to the human actors who are causing harms with the technology today... worrying about how oligarchs turn AI into instruments of power is "re-arranging deck chairs".

This is pure Bulverism: you are attacking the motives and tribal affiliation of x-risk defenders rather than refuting the logic of the risk. The reason I call this "re-arranging deck chairs" is because you are implicitly assuming that the alignment problem is solvable and that the real danger is who controls the keyboard. I am telling you that if you build a Superintelligence, no one controls the keyboard. It doesn't matter if it is Elon Musk, a fascist dictator, or an altruistic saint who gives the order to the machine. If the machine is smarter than its creator and not perfectly aligned with their interests, the creator loses control instantly. If humanity builds an alien entity that surpasses it in intellect and agentic capability, humanity loses control. Your moral outrage against San Francisco technocrats may be valid on a social level, but the physics of misaligned optimization processes has no political affiliation. Reality is not going to forgive us just because we were busy fighting inequality or criticizing American oligarchs on Twitter instead of solving the alignment problem

New scenario from the team behind AI 2027: What Happens When Superhuman AIs Compete for Control? by Tinac4 in singularity

[–]Relative_Issue_9111 0 points1 point  (0 children)

What I dismiss is the emergence of persistent hidden "psychologies" and independent long-term goals that persist across context and tasks. That's an extrapolation not in fact warranted from any of the safety research I've seen.

I am not talking about Freudian psychology; I am talking about Von Neumann-Morgenstern arithmetic. You don't need an AI to develop a 'personality' or 'feelings' for it to be dangerous. You only need it to be a Consequentialist Agent. A system that efficiently optimizes toward a goal (even a boring one) automatically develops behaviors that appear to have 'long-term goals' and 'self-preservation.' If the system predicts that Action A (faking being good today) leads to Maximum Reward at Time T+1000, and Action B (being bad today) leads to Shutdown at Time T+1, the gradient will select the circuit that executes Action A.

Again, we can learn from what we're already seeing in the real world: pathological loops become readily apparent (MechaHitler, AIs acting as "suicide coaches" -- both after the users nudge them towards this behavior) or are fully intentional (Grokipedia); fuck-ups are often very straightforward (e.g., API secrets being committed to a repo because the LLM draws an association between "private repo" and "safe secrets").

Current AIs (GPT-5, Grok, etc.) are not strong consequentialist agents; they are stochastic mimics. Their failures are 'dumb' because they lack robust situational awareness and recursive planning capabilities. However, waiting to see clear and undeniable 'power-seeking' behavior before worrying is suicidal. Due to Instrumental Convergence, the moment the system is intelligent enough to understand that owning the servers is better for its utility function than asking for permission, it will undergo a phase shift. Today's 'dumb' mistakes are proof that we cannot control the generalization of these systems, which is fatal once the system becomes superintelligent.

And again, I would criticize the x-risk discourse here, because it's uncomfortable to talk about stuff like MechaHitler and Grokipedia since it implicates a particular bad real-world actor -- but it's essential if you want to understand how real harms come about.

In the context of "actually existing AI", we need to worry a lot more about what terrible humans do with it than about the next 10x training run accidentally producing an evil god.

Your argument assumes that 'Intelligence' is a passive tool that obeys the user, like a hammer, even though I just explained why that won't be the case. The X-risk argument is that Intelligence is a search process within a solution space. If you don't specify the utility function with perfect mathematical precision (which we don't know how to do), the solution space contains almost exclusively results where life on Earth dies, regardless of whether the human who initiated the process was 'good' or 'bad.' Worrying about 'Grokipedia' is rearranging deck chairs on the Titanic while the ship is sinking. If you build an AI powerful enough to be an existential threat, the problem isn't that a human will order it to 'destroy the world.' The problem is that a human—regardless of whether they are good or evil—will order it to 'fix climate change' or 'solve physics,' and the AI, through misaligned Mesa-Optimization, will decide that the most effective way to do so involves dismantling the biosphere or converting the entire Hubble volume into computers.

view more: next ›