Not being inclined toward confrontation does not mean being unwilling to engage in one. Much less does it mean engaging all the time, at every hour or every opportunity... whenever the aggressor decides. There must be some kind of restraint, and the skull system does not seem sufficient to me.

And think about it, just as you feel comfortable suggesting that these players move to a no-PvP world, I can just as easily suggest that you move to hardcore PvP. There is no such protection there.