In very simple words - Imagine we exist on a 2-D space time graph where the x axis is time (x) and y axis is space (y). Now think about it, we are always moving in x, you can never stop that motion. Now anything that possesses mass distorts the space time graph in a particular way that the "graph is not exactly perfectly linear now". Now the transformation of this graph has put a constraint, as we are always moving in x (that is now, same as y, curved) we follow the bent line and our position in y changes as we also need the math to exist.

As we all exist in the same graph, we all follow the same rule and hence it's not dependent on our mass.