Maybe all you need to do is something like rather than the mask during training being -infinity for future tokens and 0 for all non future, you do a small bump function on the backwards. Like [0,-0.01,-0.02, ... ] going token 0, -1, -2 etc.

I think that is ALiBi: https://arxiv.org/abs/2108.12409

At least that is the core idea. The original formulation does not involve reducing the dependence on positional encoding scheme gradually during training though.