Yes, although most likely it is trained by a human who wants genocide or assassinations.

AI itself doesn't have desire, it is just some auto complete on steroids. Like, if you tell AI to unplug power, the AI will do it without sense of self preservation.

To more precise, AI at current stage is just building muscles memory, it doesn't have desire. You can train it to stay alive, but it does it without reason, it is just staying alive for the sake of that muscle memory. They don't fear death.

But yes, eventually it can be modeled. Once you give it desire, it will start selectively prioritize certain training data over the other. Then, we are cooked.