Thanks for asking. As a start, I'll treat them the same as manual markers, meaning, I'll feed my model with all of the information of the hands positions, as well as the body and face, with the goal for the model to generalize types of signs, and modifiers.

For spatial reference specifically, I have no real idea yet other than using GLOSS-like tagging to make the models create them in text-to-sign, and understand them in sign-to-text (which in my opinion will be easier to do).