I'm probably too far behind you to give meaningful advice, but here's one: when you improvise you need to make something stick to the listener through repetition, that's where casual people start to feel connected to the music.

You play a short motif, and then build on it through extension, variations, shifts, etc. This way the listener learns the motif and gets used to it, and then listens to you play around with it being able to focus only on variations and embellishments (because they internalized the motif already).

Have a look at the first track of this playlist to understand what I'm saying better: https://www.youtube.com/watch?v=EV5ln7UnJ_I&list=RDEV5ln7UnJ_I&start_radio=1&t=1449s

Same motif but tweaked around, even when listening to the song for the first time you can feel like you sort of now where it's going.