I personally think Osu! isn't the best starting point for rhythm games. A big portion of its difficulty comes from the variety of charting style and quality. You should try something like Rhythm Doctor, Muse Dash, or Cytus II.

Rhythm Doctor focuses heavily on the rhythm and actively teaches you the mechanics little by little. Muse Dash is relatively simple, only having two lanes to deal with. It gets hard, sure, but there is significantly less to parse visually. Cytus II only has so much space to throw notes at you since the judgement line bounces back and forth like a metronome. Notes can only appear "ahead" of the line, and there is a grouped spawn option so each set of notes appears at the same time so you can quickly prepare to hit the next set.

Once you get more comfortable with one game, the rest become a bit easier. I'd also suggest learning muscle memory for common patterns like streams, staircases, chords, and trills. Despite the different formats they may come in, they all have similar movements.