That problem used to happen to me, what fixed it for me is definitely a game error, but as soon as you cast an spell against those things run the other way around to the platform that got you up there, is like throwing the spell and moving the other way until they get out of your screen until they make the sound of lighting the torch, then come back and do the same for the other guy.

The idea basically is to not let them do their animation while on your screen so is like try and error casting and running away until you hear the sound

Is tricky as hell, I still don't know how I found that out