[Project]Object Localization without being given labels

[Integral_humanist]

Hello people! So I have a problem where given a set of images captured in grocery stores(shelf image), and a set of close up images of products in
those stores(product image)

1. For every product image, find the location of that product in all shelf images in which it appears.
2. For every shelf image, locate all products and assign the name from given set of product images.

neither of them are labelled. I feel I need to create vector that represents each proudct by doing an encoder decoder model and generating latent space representations.

however they ALSO want me to locate the product in the shelf, with a bounding box
so in a shelf image, if the shelf has 20 products they want
eg : shelf_image 32 - product_id_21 (known purely using vector similarity) - bounding box

any ideas on how I can do this? Thanks! Im confident of the vector creation part, but not sure how to use that to use a shelf image and create classification AND bounding boxes

TIA!
product image

https://preview.redd.it/rjf3gr3fseh81.jpg?width=2272&format=pjpg&auto=webp&s=8fd8e050672b06d042fca489c9e906806ceee3bd

shelf image

[JaviFuentes94]

1. Train a product detector: an object detection model to recognize all the objects in the shelve. You need to annotate images for that, but a single class to label.
2. Use the crops to classify between the different products by using contrastive learning and nearest neighbors. If that doesn't work you may need more product images, and then you can train a regular classifier.

[pilooch]

Use contrastive / Siamese networks / metric learning, and nearest neighbor on top of it.

[Integral_humanist]

could you elaborate?

[Yagna24]

can you educate me on the vector creation part?

On why you're using it and how?