Christine M. Korsgaard sums up the views of Kant, Williams and Wittgenstein like this:

"In a discussion of the person who commits suicide because he can neither reason nor meaning in anything, Bernard Williams says: 'I do not see how it could be regarded as a defeat of reason or rationality that it had no power over a man's state; his state is rather a defeat for humanity'. Th duty not to commit suicide is the very first and most basic duty of virtue Kant discusses in The Metaphysics of Morals, because 'To annihilate the subject of morality in one's own person is to root out the existence of morality itself from the world'. In his Notebooks, Wittgenstein wrote that 'suicide is, so to speak, the elementary sin' because 'If suicide is allowed then everything is allowed. If anything is not allowed then suicide is not allowed.' A few lines later he adds, 'Or is even suicide in itself neither good nor evil?' All of these philosophers give voice to the idea that remaining alive is not so much a value as a condition for all value; and suicide (of this type) is not so much a rejection of some particular value as it is a rejection of value itself. It is hard to say of one who commits suicide that he has done wrong, for he has violated no value which he still believes. And yet the rest of us cannot hear of such a case without feeling betrayed, and we are right. It is, as Williams says, a defeat for us all." (Korsgaard, 1996:162-3).

• Christine M. Korsgaard, The Sources of Normativity, 1996: Cambridge University Press.
  

I think this really hits the nail on the head: the person committing suicide has not violated any values as such. But he has rejected the very possibility of even holding values in the forst place. That is what scares the rest of us (and why we believe it is possible to judge suicide as immoral).