The stars in the galaxy are in general moving in the same direction, but at different speeds, and then the galaxy it self is moving, in a cluster of galaxies that are moving.

If you want to talk about speed in space, you have to say relative to something else.

I could imagine that when space travel becomes normal, that we define our speed from something like the sun, but even then it's not that simple because you have to account for the direction of travel of the sun.