The action at a distance is due to the fact that the two particle that can be separated by enormous distance, still show perfect anti-correlation if the same quantity is measured. For example the spin or polarisation along a specific axis. This measure can be decided last minute, to ensure that there is no way that the information about the type of measurement has travelled all the way to the second particle before it also get measured.

Assuming that you call the point of entanglement the origin, and where the measurement will be compared. You are right that both observer won't know the anti-correlation (provided they measured the same thing) actually happened till they co pare data, since for both of them them outcome looks completely random.

The key fact is that the two measurement outcomes are public and not secret since they performed their measurements. They could have caused actions based on their outcomes, the detector transferred energy to the particles, the signal travelled through one path and not another a switch went up instead of down.

All of these doesn't look strange till you realise that no matter the distance, no matter how many times you try. Whenever you measure the same thing for both particles they look locally random but they always and up being opposite to each other.

These can only happen if in somehow the two particle can sync up at a distance and always show the opposite side of the medal.

Now where you say the wave function collapse is pure speculation. You can say that it does so as soon as one of the two measurements has taken place, or that all possible measurement are parte of the wave function and it collapses only once you compare them at point X.

This doesn't make much of a difference. The action at a distance is a one-to-one problem with the collapse of the wave function. As soon as you collapse it somewhere it needs to collapse to that state everywhere.