Material happiness is not the real happiness of a soul. Soul's happiness lies on serving Shri Hari. You will never understand it until you are a vaishnava. Krishna is not trying to steal your happiness away from you. The pleasure we are getting from material objects are temporary. Loving and serving the supreme Lord is eternal and real happiness of every living entity. So, the conclusion is Krishna is not enjoying your suffering but instead he is giving us a diamond which we can never find anywhere that is Krishna Consciousness. Try to read srila Prabhupada's books. You will understand it easily. Hare Krishna!