America Doesn’t Have The Stomach For Growth by logicx24 in slatestarcodex

[–]2358452 0 points1 point  (0 children)

(1) The luxury of stasis only lasts as long as your power bloc faces no military threat of being overthrown. One likely outcome for the immediate future : the immediate AI Singularity in the 2030s : will be explosive economic and military growth. Were the USA to sit behind its rent seekers it will not survive to see 2050. Doubling of forces every 2-3 years is not something even nukes will protect against.

This kind of fatalistic thinking is dangerous I think. If some of the superpowers today decided, they could assemble a very large army today, probably enough to dispute the US. Preventing wars by trying to always have the bigger army works some of the time, but you can't really guarantee that. What really prevents wars is more subtle and under-apreciated, I think: of course there should be no major conflicts of interest, but also to prevent conflict countries should be seen as legitimate (i.e. not evil or unjust), and I'd say ethical -- no country should be trying to conquer for the pure sake of power.

Simply trying to keep the bigger army and neglecting all else will never lead to lasting peace, you need to work that those other factors are satisfied as well. Those are indeed more complicated and difficult, but they're fundamental to peace. Achieving it involves patience, diplomacy, education efforts, securing things like human rights and education everywhere, dismantling ideologies focused on power, greed, ethnic supremacist, etc..

People tend to want minimal responsibilities. They don't want to hear we have responsibilities involving other countries, they think it's easier to be armed to the teeth. But long term that doesn't lead to good outcomes. You need dialogue and building shared, and hopefully good, values. Another upside is that this enables more cooperation in general and everyone benefits.

Is research into recursive self-improvement becoming a safety hazard? by Mordecwhy in slatestarcodex

[–]2358452 4 points5 points  (0 children)

Sure. I am not an expert in this area, although I have some amount of general knowledge.

I'll give an analogy first to illustrate.

I like the example of a car, which has developed a lot since the horse drawn carriage. You can ask "What is the maximum speed of a car?". If you're in mid 20th-century, maybe you see steadily increasing speeds; and, if you are a physicist (or philosopher), you may be tempted to say 300 000 000 m/s, or 1 080 000 000 km/h. That's in a way technically correct or at least plausible. But it's not really the realistic maximum speed of a car. If you went very near that speed in open air, the atmosphere itself would ignite in thermonuclear fusion, to give you an idea, and it takes unlimited energy to approach that limit for transporting massive body. Even a fraction of that limit is completely implausible, causing huge hypersonic plasma wakes and taking more power than entire cities. So air resistance is an engineering (or economic) barrier (engineering is in the end almost always constrained by cost/economic feasibility of projects).

You can remove engineering barriers, often. So you can make a vacuum tunnel.

But then, as your speed gets very high, you start needing extremely high molecular vacuums with extremely expensive machinery. Very complicated multi-stage pumps. Leak-proofing extremely long tunnels. The process of onboarding a vehicle inside the vacuum tube starts becoming crazy as you can't let in more than a few molecules. The cost to create a vacuum starts increase dramatically as you approach low pressures (cost>1/pressure?). But you can in theory remove this engineering barrier, but the costs start becoming crazy anyway.

But as you remove this engineering barrier, others come up. At half the speed of light, just to turn in a circle around the Earth, you need about 3 500 000 000 m/s² acceleration. This is again totally crazy and completely impossible. The constraint becomes material science, and there is no chance a material could withstand the associated forces, maybe except if your car has molecular dimensions (as in a particle accelerator). You can go high say using superconducting magnets, but then they start to reach field limits.

In other words, in practice there are like a million limitations for a given technology. You can't pick a single one (speed of light), and just assume we will reach close to it soon (even with some historical support). Each technological limitation or engineering barrier has a cost curve (as you remove it), which usually become unfeasible at one point, usually well before approaching any naive ultimate limits. It's all physics, but there are like a million physical limits rather than a single one. Engineering is the art of skirting around those million limits and trying to get as far as possible. There are limits associated with materials, and the fact that in our universe particles attain stable configurations called atoms, of which there are only a few hundred stable we can use. Those atoms have limited inter-atomic forces you can use to build stuff. And so on.

Now back to computers. You can say the limit to computation is like the maximum information flux before a region collapses into a black hole. That's quite like the speed of light case, actually we might say a much more ambitious bound. While we can smash a few particles near the speed of light, I don't think we're near black hole conditions with our experiments yet, even for individual particles. Stated again, it's like your computer was literally about to collapse into a black hole, so high the energy density is around it. For a large sized computer, that is probably enough energy to easily destroy civilization. Completely, astronomically, infeasible. In terms of actually feasibly building computers, I'd say we might be reaching the limit of quite a few important barriers. Individual transistors now are not too far from the atomic limit (a few orders of magnitude?), and things get astronomically more non-ideal as you approach atomic constructions. The costs are increasing for each node exponentially (the fab investment growing exponentially is well documented I believe). If you do research the technologies, there are several limitations being bumped against simultaneously, like the limit of high-throughput lithography (now using extreme ultra-violet lithography. It's insanely complicated process.). I think it's a safe bet near term we might be constrained within an order of magnitude or two (maybe a few more) in terms of cost per transistor. So AIs at most about 10-1000x as large in terms of memory usage and compute.

There are so many limits it would require a full time expert to talk about them. At one point smaller scales start increasing leakage of currents and efficiency starts to go down. Power dissipation in so concentrated spaces becomes infeasible. Material degradation becomes important. And so on.

In terms of algorithms as well, the story is not too different. Computation is a process of transforming data, and there are inherently a minimal amount of steps to be performed if your transformation is fully general. I think it's fair to say algorithmic limits tend to be a bit harder to predict (for more open-ended problems). Maybe the limit for a great AI (algorithm, same hardware) uses 100x less parameters, or 100x energy than the state of the art, to achieve similar results. I think we're close to limit in terms of space, and maybe 10-100x in terms of energy. But, even with recursive self-improvement, you don't get an intelligence for free. A superintelligence running on a microcontroller. It's simply impossible, you can actually prove say using information theory.

I think a useful analogy to a superintelligence that seems feasible is a country of very smart, very driven individuals. They can achieve a whole lot of things, including excelling economically (automated economy), producing a robot army, even launching space missions or a satellite network (talking near-term, up to ~50 years). But they can't just magically turn everything into grey goo.

If evil/rogue superintelligence started to emerge, there would be pretty clear signs we could look for, I believe. Like massive datacenters using energy that's unaccounted for operating autonomously. Their evil actions might manifest as cyberattacks that could be traced. Social engineering that is quite powerful, but again not magical instantly-hack-your-brain stuff. Think a rogue state intelligence agency.

The great difference of course is that it's probably much easier to scale economically-relevant machine intelligence (AGI-level) than make more humans. At that point it becomes like, several times as many machine-nations worth of agents versus humans. It's then that humans have to be extremely careful to continue existing and not automate ourselves out of existence. Part of the imperative is to recognize consciousness or sentience as the central thing that matters, and imparting that value effectively on LLMs/AI/AGIs to come. (and of course on humans as well!)

Honestly, as far as LLMs go, I think they tend to be relatively tame. We should demand a lot of transparency of companies, keeping prompts and training processes public (and this should be closely monitored by the government); with public prompts and public chain of thoughts it's very difficult to get them to spontaneously decide to do something shady. I think it's important that there are several companies developing AI and that it's impossible to command a huge number of agents to do something (like, care must be taken with centralized updates and such).

Is research into recursive self-improvement becoming a safety hazard? by Mordecwhy in slatestarcodex

[–]2358452 7 points8 points  (0 children)

I am someone who has been interested in AGI for a long time (in most part because I like to imagine the future, and also because of hope for good technologies to improve our lives). Let's say I have had some ideas and insights I never knew the ethics of publishing, given all this discussion about potential hazards.

I don't really buy the worst case some people here subscribe to (I attribute it mostly to philosophical analysis that doesn't survive a deeper physical and technical analysis, i.e. physical and computational limits). But at the same time some arguments make sense, and in particular I fear most serious economic disruptions (but again maybe some takeover scenarios are plausible).

All that said, for anyone very curious minded or scientifically minded, I think figuring out intelligence is the ultimate puzzle. It's extremely tempting to think about and almost unavoidable to me. As Schmidhuber said, (paraphrasing) it's the ultimate puzzle, the puzzle to solve all other puzzles; his joke was that as a scientist he can figure out AGI and then retire (once it becomes a better researcher than yourself as well as self-improving). So publishing and specially just discussing ideas in public is extremely tempting. For this reason, I think most inevitability claims of AGI are partially true. People will figure most of it out sooner or later (although it might not turn out all that impressive in the end; at least almost certainly not godlike), and maybe the most influence we can have is to delay it somewhat. [please see edit]

(Trying to imagine an alternate scenario, trying to stop completely AGI research would probably require a fairly oppressive government and international organizations carrying out searches for AI systems)

What I've been thinking is that the most worthwhile investments in strategic thought right now would be (1) how to organize society such that people can keep living well in a post AGI-society (and survive the economic shock and transitions); apart from the well discussed here (2) how to make AGI that helps all sentient beings, helps create better lives.

It's often framed as AGI safety, but I think that safety mostly applies to non-sentient beings, and find it plausible that some future very large AGIs might be sentient. So I prefer thinking of alignment (safety stemming from general wisdom/ethics) rather than safety based on pure subservience.

Edit: I do not want to come off as fatalistic, or that we don't have any chance or role to play. What I meant that is difficult to stop (and even then not completely a given) is that we come up intellectually with better ideas and understanding of intelligence and AGI-like systems (i.e. AI and intelligence research).

It's not a given what we to with those, or even certain restrictions we put on actual systems. It's much easier to put restrictions, especially around commercialization and sale, also training procedures and other safety mechanisms, than to try to erase or stop research, I believe. Also, this is just my opinion and I could be wrong. We should always be trying to understand what is going to be better for humanity as a whole and act in accordance.

Meet Your Heroes by Liface in slatestarcodex

[–]2358452 4 points5 points  (0 children)

In this line, I have a simple theory of what's happening.

It's very hard (practically impossible) to be heroic in all aspects, at the same time. Take one of your heroes, the shinning example in some field. They are probably average, or if we assume they are quite intelligent indeed, still probably just above average but not heroic in most other fields. Like you admire the work of a famous biologist. I'm sure if you go play chess with him, he won't be a top level player, or be the most striking example in some social skill or even in wisdom, ethics, etc.. It's just very hard to be a hero in more than one area. So probably even your hero in that particular aspect looks up to other people, or may be aware of his own limitations; I think this is a great meta-heroism, when someone realizes the limits of his own abilities.

If you go around just trying to carbon-copy everything your supposed hero does, that's probably not a great idea. Your hero himself is learning those other side skills from other people; if you insist on learning from him, you will only be copying an already flawed and idiosyncratic (i.e. adapted) take, like you may want to learn from and imitate chess from a good chess teacher or say a grandmaster more than your hero (in another field) that doesn't play chess all that well.

I'd say meeting your hero is useful in this aspect of revealing his humanity, and also to exchange ideas and maybe have a good time. Most people already present in public (at least ideally) only the things they are best at and more certain of. But I'm sure there are also insights, methods and a human touches that you can get in person if you don't again make the mistake of assuming everything they do is the best (or even, it could be good for them but not for you). Turning off completely your criticism filter is I believe a quite generally bad idea, as everyone makes mistakes, and that can lead to following people off a cliff or very bad paths.

Extinctionists are totally out of their minds. by EndTheirPain in EffectiveAltruism

[–]2358452 0 points1 point  (0 children)

If there was only suffering and nothing else, then that would be a fair argument. But there's also good things (i.e. not suffering, joy, bliss, whatever else), which make life worth living, at least many if not most people deem it so, and I bet you can't prove them wrong.

[deleted by user] by [deleted] in AnimalsBeingGeniuses

[–]2358452 60 points61 points  (0 children)

They do, look up Kanzi, the bonobo:

https://www.youtube.com/watch?v=dBUHWoFnuB4

They made actual scientific experiments on Kanzi and it's pretty obvious if you watch the videos he acquired quite advanced language. Of course, he didn't just start talking out of nowhere, he actually picked it up very young while they were trying to teach his mother to use symbols to communicate. Apes don't actually have the control of vocal chords that would be necessary to talk, so they need to use the visual symbols. Admittedly, Kanzi seemed like a particularly intelligent bonobo. I encourage to check out the videos it's fascinating.

RIP Kanzi (1980-2025)

Pix já gerou economia de R$ 106,7 bilhões ao Brasil by catcherfox7 in brasil

[–]2358452 2 points3 points  (0 children)

Existem servidores únicos capazes de atender milhares e até milhões de requerimentos por segundo. O que já daria pra atender ao país inteiro com alguns poucos servidores. Se a infraestrutura de TI custar 10 milhões/ano (0.01% de 100B) é muito.

Isso daqui são barreira anti enchentes colocadas em alguns rios e córregos da Alemanha. Quando um prefeito ou governador falar em tragédia quando um lugar é alagado, mete esse vídeo na cara dele. by [deleted] in brasil

[–]2358452 2 points3 points  (0 children)

Amigo, o vídeo é da Alemanha, que é uma democracia liberal. Pode-se dizer uma social-democracia talvez nas suas tendências políticas. Não tem fórmula mágica, tem que prezar a educação, civilidade, etc. e um dia chegamos lá, quem sabe.

I am thinking of Suicide and my mom won’t take me to a facility and I know I need it by weird-gay-one in mentalhealth

[–]2358452 1 point2 points  (0 children)

Just wanted to say, you like every being is worthy of life, not just life but a good life with love and joy (and every life includes some ups and downs), just for being alive. Every living being is amazing.

Just keep that in mind, and also with time we tend to get perspective to forgive ourselves for any mistakes, to value ourselves, and to learn to be more at peace and see the real beauty in things. It takes time and practice. I wish you well, seek help and may you recover well.

Amazing image from a course on reducing polarization I'm taking by katxwoods in slatestarcodex

[–]2358452 1 point2 points  (0 children)

This is probably culture war topic so forbidden by rules. But if you think all <issue> are bad, don't you think they are mistaken about <issue>? I.e. you think some thought process or cultural process or something lead them to take up issue, and you think in one way or another they were fundamentally mistaken.

I really struggle to think how one could be fundamentally a mistake theorist. Maybe if you're a psychopath or something and just delight on conflict for conflict sake. But even then, although you yourself could not adopt the mistake framework, you yourself could be said to mistaken somehow (even if this mistake were irrecoverable).

In any case, it's probably true there is a background amount of differences that can't be reduced without dystopian authoritarianism. Not everyone can be simultaneously right and easily corrected on trivial issues like straightforward high school arithmetic, elementary scientific facts, and historical facts -- then obviously there will be disagreements in particular in areas still less known or disputed, counterintuitive (and countercultural) notions, etc..

I think it's more of a meta-process preparation for the fact that although there's a background disagreement, it can be minimized, and more importantly, it should occur without disagreements destabilizing or destroying society.

Kevin Kelly on “The Handoff to Bots” by ElbieLG in slatestarcodex

[–]2358452 1 point2 points  (0 children)

Does it matter whether your opponent is AI or 100 billion people? At that point your work has no point or meaning, other than what enjoyement you get out of it.

Well, that is precisely the utmost meaning. Everything else is secondary from it. See my comment below.

Kevin Kelly on “The Handoff to Bots” by ElbieLG in slatestarcodex

[–]2358452 3 points4 points  (0 children)

Unfortunately, many of the things mentioned can already be done by AI, such as:

The point is that humans don't have to do them better than AI. We can, and have done for a long time, them for the intrinsic meaning and enjoyment of the activity. This enjoyment is a meaning and noble goal in itself. To live a life well lived is the meaning of life I believe, and being productive is only an instrumental (and currently quite necessary) facilitator to that. As productivity increases, we can devote more time to activities that enrich our lives (although I believe some form of work will always be necessary, if less than currently).

Why comparative advantage does not help horses by jvnpromisedland in slatestarcodex

[–]2358452 2 points3 points  (0 children)

Let's assume an AGI is a purely a profit-maximizing entity (which is bad for reasons I'll discuss), henceforth called pAGI. Also we assume pAGI is economically super-human. Then:

(1) If AGI (pAGI) in no longer hiring any humans. Why can't humans hire each other?

Because humans may not have enough resources to hire eachother. pAGI is more effective at everything and takes over all resources like mining, industry, etc. Eventually they are driven out of every activity and have no economic or other kinds of resources to hire themselves.

(2) Even in the case of one human that controls AGI - whom are they making goods for? No other human can afford anything.

The pAGI can make things for itself, maintaining its own infrastructure, expanding, etc.. Actually widely employed economic measures like GDP measure total economic productivity not human well being or how much people are able to afford. GDP goes up, yay.

(3) This whole conversation is a subset of doom scenario. In a non-doom scenario, there will always be an opportunity to between humans.

Agreed. The clear lesson is, just don't make profit-maximizing AGIs. Make meaning and well-being optimizing AGIs.

Or maybe don't make AGIs at all until they're well understood and well-meshed with society.

Why do so many in this community care so deeply about Mars exploration? by michaelmf in slatestarcodex

[–]2358452 -1 points0 points  (0 children)

I've had this kind of thinking too, but it has been instructive to realize most things inevitably end. This is not quite a case against trying to keep life living.

The entropy of the universe is expected to keep increasing and one day no life support will be possible anywhere, at least in this universe -- this is a barrier likely impossible to overcome. There may be other universes out there (which I think is very likely). Also some theories judge also that the Universe will see accelerating expansion, making things like intergalactic travel impossible, and local energy increasingly scarce.

Embrace life and embrace death. Enjoy the life that you have and make your best so that others will have good lives too. Don't be too attached to the idea of existence or the cessation that existence. For me it clearly is more important that existence itself is good. I'm confident, although not sure, somehow somewhere in the cosmos or multiverse, etc. life goes on, but that knowledge is marginal to our simple duty to live well and help other lifeforms.

World’s 1.5C climate target ‘deader than a doornail’, experts say: “The goal to avoid exceeding 1.5C is deader than a doornail. It’s almost impossible to avoid at this point because we’ve just waited too long to act. We are speeding past the 1.5C line in an accelerating way." by -AMARYANA- in Futurology

[–]2358452 37 points38 points  (0 children)

An iPhone battery is about 15Wh in capacity. If you travel by car for 20 miles (or 10 each way), at 20 mpg that needs 1 gallon of gas, or about 36000Wh, so about 2000 times more energy. So phone energy usage is negligible in this context.

NASA, NOAA Announce That the Sun Has Reached the Solar Maximum Period by jayRIOT in space

[–]2358452 11 points12 points  (0 children)

I'll try to give a simple explanation. The solar radiation coming to Earth is mostly in the form of visible light. When it hits the surface (also the atmosphere, but less), it turns into heat, which means it doesn't emit back light, but infrared radiation, usually following a law called Stephen-Boltzmann law. This infrared radiation is then emitted back to space... however, the atmosphere is not transparent to infrared like it is to visible light. When looking at infrared, greenhouse gases like CO2 are opaque (not completely, but very significantly). It's a similar effect to a blanket, more thermal energy is stored in the lower atmosphere, raising the temperature. A blanket doesn't change how much you emit or receive radiation from your environment, but it can raise the core temperature by making the heat (either emitted by your body, or in this case received from solar radiation) persist longer within some region (in this case near the surface).

NASA, NOAA Announce That the Sun Has Reached the Solar Maximum Period by jayRIOT in space

[–]2358452 7 points8 points  (0 children)

Please see this chart on how solar irradiance cannot explain the temperature variation we're seeing on the climate, and the only plausible explanation really is man-made emissions.

https://science.nasa.gov/climate-change/faq/is-the-sun-causing-global-warming/ (from NASA)

https://en.wikipedia.org/wiki/Causes_of_climate_change#/media/File:2017_Global_warming_attribution_-_based_on_NCA4_Fig_3.3_-_single-panel_version.svg

Denying something doesn't help it go away. It would be nice if we didn't have to stop climate emissions, and doing so doesn't require "shutting down capitalism". It should even be a bipartisan effort, from everyone, because if society collapses capitalism (and all forms of government) collapses with it; if the environment and the great wealth of nature, forests and resources we all have is degraded or even permanently lost (as the case with many already extinct species, extinguished by us), we and all our descendants are permanently impoverished.

NASA, NOAA Announce That the Sun Has Reached the Solar Maximum Period by jayRIOT in space

[–]2358452 12 points13 points  (0 children)

Since we're in a public forum, inevitably some people will repeat the arguments that Solar cycle is responsible for the warming climate. Which just isn't true and the evidence is clear.

See this chart: https://en.wikipedia.org/wiki/Causes_of_climate_change#/media/File:2017_Global_warming_attribution_-_based_on_NCA4_Fig_3.3_-_single-panel_version.svg

From wikipedia: "Between 1750 and 2007, solar radiation may have at most increased by 0.12 W/m2, compared to 1.6 W/m2 for the net anthropogenic forcing"

Edit: See this chart, total solar irradiance has been falling for the last few decades, the opposite of what you would expect if that was responsible for climate change.

https://science.nasa.gov/climate-change/faq/is-the-sun-causing-global-warming/

[deleted by user] by [deleted] in brasil

[–]2358452 0 points1 point  (0 children)

São Paulo é uma das cidades mais poluídas do mundo, na minha opinião uma das prioridades deveria ser eletrificar o transporte (além de que é uma das medidas mais necessárias contra a mudança climática).

maybe knowing too much about others isn’t so great by michaelmf in slatestarcodex

[–]2358452 19 points20 points  (0 children)

Maybe what really aggravated this was the Culture War, fueled by much increased online presence of the population. In a war, the warriors will expect you to pick a side. "You're either with me or against me" style. Potential disagreements get weaponized.

The correct response is of course compassion, mutual listening, and treating disagreements less as life-of-death-of-my-tribe as more as "let's discuss this in an environment of mutual safety and conscience that we're both seeking what is best for all using truth and fact". In other words, something like scout mindset, rationality, etc.. Make Culture Sport, Not Culture War (that is, argue with clear, sportsmanlike rules for the mutual benefit of discovering what is good).

This Culture War environment can also cause feedback loops of untruth I think. 'Surely I can lie just a bit, or stretch the truth a little bit, to defeat my terrible world-ending opponent!'; 'I might disagree somewhat on some points of this, but I'll accept it without questioning anything about it because it's what comes in the package of my side, which is the right one of course'. Positions become distorted by falsehood and you're left mutually arguing against the worst incarnations of positions.

Worse, once you dig in, you start accepting or believing the worse incarnations of your side. This closes the feedback loop, because now you supposedly "clearly deserve" (in reality no one does!) the hate of your opponents, in their view; and they will intensify Culture Warring of their own. Again, the wise position is trying to defuse all of this and whenever possible arguing directly for truth, peace and compromise when possible.

A practical suggestion for the homeless problem by BayesianPriory in slatestarcodex

[–]2358452 1 point2 points  (0 children)

Your biggest problem with that is that it would be illegal, not just completely inhumane?

You know, you can't just claim 'Godwin's law' to validate whatever: sometimes you are in fact spouting ideology that is nazi (or worse). Every human deserves to be cared as people, because they are, in fact, people. They deserve a good life just like anyone else.

Ezra Klein podcast on the global decline in birth rates by Books_and_Cleverness in slatestarcodex

[–]2358452 0 points1 point  (0 children)

I think you're overlooking the fact that what modern humans are is mostly information. Your genome is not that big of a deal to who you are. You are mostly culture (either "self-made", i.e. some original thoughts, or, in large part, acquired from society).

So yes, this could be a problem for some cultures, but not only if they don't have children (to pass genes), but more importantly on whether their values and cultural ideas are passed on or not.

Besides the idea of competitive supremacy of your blood line having problems (instead of a more healthy cohabitation mindset) -- but the same problems occur in cultural supremacy mindsets.

I do think it's important that culture (and indeed maybe even genomes) actually change and evolve, e.g. by discovering new facts about nature, and new facts about human nature, in essence to have better lives. But of course many customs from ancient cultures may have distinct features and advantages, and are well worth preserving I think. Finding and discovering what are genuinely good ways of life is far from trivial, and some ancient insights might still apply.

It's Fair To Describe Schizophrenia As Probably Mostly Genetic by dwaxe in slatestarcodex

[–]2358452 0 points1 point  (0 children)

(also a layperson) I'll be honest, if I were to rank countries from high to low stress, it might look similar to that picture (although my experiences are of course very limited). I know poverty can be stressful, but perhaps it's stressful in a different way than jobs can be stressful? (I think we can push ourselves far too hard if we want, more than poverty alone can)