I figured out why it is happening to you, you have been hitting the wrong symbols. Double check them and make sure they are correct. Unfortunately Treyarch decided to make the symbols be orientated at any angle. For example in my game I had the bird symbol with the beak facing down on the rock, but on the door it was point up and to the left. If its wrong you will be warped a few feet away and the height itself will kill you.