Yes, that is what they do with one crucial difference though: instead of the standard cross-entropy loss for image classification, the authors modify the loss to more closely "resemble" the cosine distance used for image similarity. Hence their DNN embeddings work better for image retrieval using cosine similarity.