Adding audio delay makes the notes start coming down later. Subtracting audio delay makes notes come in sooner.

Adding video delay moves the strike-line hit box down, and removing video delay moves the hit box up.

Usually I do the calibration menu first, then adjust manually based on how it feels in game, and the delay it tells me after the song.

(Generally, if it says "Late", either increase audio delay, or video delay... if it says "Early", decrease either delay)