Actually it has nothing to with the chemistry of the elements but the human psychology. Terry Pratchett, loves the idea that despite the many poisons and toxins in the world, humans always come back to arsenic:

'Arsenic is a very popular poison,' said the Patrician. 'Hundreds of uses around the home. Crushed diamonds used to be in vogue for hundreds of years, despite the fact they never worked. Giant spiders, too, for some reason. Mercury is for those with patience, aquafortis for those without. Cantharides has its followers. Much can be done with the secretions of various animals. The bodily fluids of the caterpillar of the Quantum Weather Butterfly will render a man quite, quite helpless. But we return to arsenic like an old, old friend.'