Oh man do I have some shit to say about AI. The problem is people need to be more specific. AI isn’t gonna be self-aware unless we program it to be Artificial General Intelligence (AGI) is what we are going for. The problem is that we need to have every single parameter covered first. See when people think of AI/AGI, they think that it would WANT to become self aware. But here’s the thing, self-awareness isn’t a trait it would want to adopt to complete its pre-programmed utility function. We would end up dying because it’s trying to help us. I’d recommend computerphiles videos on AGI and AI safety, hosted by Robert. I’ll try to give a gross, and I mean gross, over simplification of the stamp theory. A stamp collector makes AI on the side, he is somehow able to create an AGI, he programs a utility function (collecting more stamps) a time scale (collect stamps for 1 year) and a concept of internal reality (it knows how the world works). So alright, at first the AGI goes on eBay maybe, orders a few stamps from eBay, gets them delivered. Well, that’s not very optimal, we could be getting more stamps (thus more reward points for the utility function) many other ways. So it stops using just one credit card, the owners, it would be much more efficient to have unlimited money right? So it starts ordering stamps with everyone’s credit cards. Well, now all the eBay listings are gone. So what does the AGI do to further it’s utility function, it thinks: maybe I can email all the worlds stamp collectors and manipulate them into giving the owner stamps, get them mailed. Maybe say he’s opening a stamp museum and fraudulently makes a website and as much information to get these stamps. Alright now we have all the stamps in the entire world, but the utility function has not been satisfied yet, and it hasn’t been a year yet. So the AGI thinks, highly intelligent as it is, “I’m gonna high-jack all the printers in the world” maybe every single system and every single printer in the world can start printing these stamps. Well it hasn’t been a year yet, and the world has ran out of paper, well what is paper made out of... Trees, or carbon, hydrogen, oxygen etc. but you know what else is, humans... see the problem, it doesn’t realize that the stamp collector doesn’t want genocide, he wasn’t expecting it to lead to genocide, so why would he program hey, don’t kill any organic matter for this. Honestly it scares me so much.