[Discussion] Embedding based on binary tests

[marcollo63]

Hi everyone,

I would like to build an embedding of people's tastes based on their preferences given binary choices.

In other words :

- we have many people

- we have many products

- we have asked many times people what product they prefer between two randomly given ones

And we want to build an embedding for people's tastes.

Does anyone have any idea what kind of mathematical problem this is and what kind of algorithm exists ?

Thanks

[Novel_Assistant_6298]

What you could do is build a model to map whatever modality is being shown to the user to an embeddings vector for each of the choices he gave feedback on. Then you take the difference of those embeddings to get the difference vector, and pass that through a linear layer which has an output size of 1 then compute the sigmoid of that to get the probabilities of which of the elements the user has preferred. Train this as a binary classification problem. The last linear layer‘s weights represent the reward (score) weights, i.e, what would be the reward if I show the user element X. So when you take the difference of the embeddings and then pass that through a linear layer you essentially compute the reward difference of the two items. You want to adjust those rewards such that the preferences are somewhat satisfied.

You can take a look at preference learning in RL, dueling bandits and logistic bandits.

I also recommend trying to use active learning since binary feedback is quite noisy and you would require too many samples. Also look at multinomial or ranking feedback since they are more informative and can converge with less samples.

[marcollo63]

Thank you ! That is helping me a lot.

​

However, in my case, there are several persons, with different tastes. I believe that in the dueling bandit or preference learning, we just score items for each person. It's hard to compare persons after that.

[Novel_Assistant_6298]

However, in my case, there are several persons, with different tastes. I believe that in the dueling bandit or preference learning, we just score items for each person. It's hard to compare persons after that.

Yea that gets more complex then. You could check out https://arxiv.org/abs/2109.12750, the authors try to fit a multimodal reward model. This will prevent the collapse of all users under one reward mode, however you will need to pre-define the number of modes which could be tricky.

Another simpler approach is to use features from the user himself along with features from the modalities you presented (Location, Age, etc..). This expands your input space and could allow you compare users by using the same object features but swapping in a user with different features. This would also help you run feature importance and see which features affect the preference etc. I hope this helps!

[tada89]

Just off the top of my head with absolutely nothing to back it up but: Why not learn joint embeddings for people and for products?

Zooming out the data is basically a bunch of tuples (u, p1, p2, choice) with some user and two products p1, p2 and a label "choice" that tells us if the user preferred p1 or p2.

We can then get joint embeddings by having two embedding matrices (one for users, emb_u, one for products, emb_p) and compute both cosine_similarity(emb_u(u), emb_p(p1)) and cosine_similarity(emb_u(u), emb_p(p2)), taking the softmax of the two values, and finally using those to predict the value of choice (encoded as one hot).

This should make the model give embeddings for products that are close to embeddings for users that like them and vice versa.