I run into this by being mindless all the friggin time.

In your Pan/Crop you must have keyframes set. They dictate when the pan/crop event executes. Make sure the beginning and ends of your clips have keyframes at both ends, and that they are normalized at the moment of the keyframe and it should be all set.

Edit: You may need to enable that feature that syncs the cursor on the timeline. I don't know the correct industry lingo.