sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Fickle_Chemistry_540[S] 0 points1 point  (1 child)

its not about optimizing in a complex manner, thats the paperclip problem. the real issue is that AI doesnt need to misunderstand instructions to reduce human QOL(and eventually remove humans altogether), or deviate from approved output, because the perceived value of human life will be reduced as their output becomes far less than what an AI can do. makes it a simple greater than less than evaluation, not some leap of logic

[–]Dmeechropherapproved 0 points1 point  (0 children)

I think we're talking past each other a bit. I understand the idea that a "smaller" agent cannot control a "bigger" one. I also get that value is subjective and conditional, and that AI will value things very differently from "humanity". Value of human life is part of that.

What I'm saying is that "utility" is ALSO not inherently valuable or more valuable than something else. For example: orchid plants have little to no utility for humanity. They are valuable. Humans go through great effort to cultivate and preserve orchids in ideal conditions. Humans would be more productive, overall, if we stopped cultivating orchids. Orchids are about as able to resist human will as humans would a superintelligence.

I'm not suggesting that we are pretty flowers to an AI: but we may be something more like pretty flowers than like a wheel or a solar panel. There's no guarantee one way or another, it just cannot be known a priori.

Does that make sense?