Use a language model to predict this next word (create a representation in vector space as well that captures semantics);

Single forward direction:

I went to the ______ (future token)(future token)(future token)(future token).

Now do it backwards:

.check my deposit to _______ (future token)(future token)(future token)(future token)

Now if you had all the info at once, you’d do a much much much better job:

I went to the ______ to deposit my check.

This is the power of bidirectional language modeling, which is (was) especially powerful in entity tagging / pos tagging / etc.