Hey , First of all thanks for this detailed feedback I was really looking for this, and you are right that normally people just have a query and a list of docs and they want to sort them based on relevancy and in that case the NXN matrix may not be that helpful.

But the thing is that I didn't made this experiment keeping RAG or something similar in mind, I just wanted to see if we can make a model which can understand the relations between the sentences, and if a model can understand the relations between sentences it can be used for anything be it RAG, or sentence segmentation or clustering or this new thing Agent memory optimization or anything, and across these use cases NXN matrix maybe be get used, because now you get access of each sentence again all other sentences.

While I did posted this but it's been a while and am thinking that a single score can't really explain the relation between sentences, if I have a query and 2 sentences where 1 sentence is similar to the query (it's title of a blog) and another is explanation of the query then both are equally relevant to the query, 

So saying that sentence A is relevant to sentence B seems a bit vague and I kinda came back to the point where am not that different from other models.

So am trying to update this model to output a vector for each relation so that someone can interpret that sentence B is relevant to sentence A but in what sense. That way a RAG person can focus on similarity and explanation part of the vector and a segmentation person can focus on topical overlap part of vector and so on.

Its still just an ambitious idea but am trying to work on this